Skip to content

fix: isp column appears to be out of order between last_seen and daily table for some products #7157

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Mar 11, 2025
Merged

fix: isp column appears to be out of order between last_seen and daily table for some products #7157

merged 1 commit into from
Mar 11, 2025

Conversation

kik-kik
Copy link
Contributor

@kik-kik kik-kik commented Mar 11, 2025 •
edited
Loading

fix: isp column appears to be out of order between last_seen and daily table for some products.

This change is to fix the location of the isp column in the last_seen view. Making this change as currently this results in dry run failure for focus_nightly, focus_beta and klar.

blocks: #7072

@kik-kik
fix: isp column appears to be out of order between last_seen and dail…
d20e6c0 
…y table for some products. Fix order of the isp column in the last_seen view
@kik-kik kik-kik mentioned this pull request Mar 11, 2025
Merged
@dataops-ci-bot
Copy link

Integration report for "fix: isp column appears to be out of order between last_seen and daily table for some products. Fix order of the isp column in the last_seen view"

sql.diff

Click to expand! 
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/event_monitoring_live_v1/materialized_view.sql	2025-03-11 11:53:35.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/event_monitoring_live_v1/materialized_view.sql	2025-03-11 11:57:45.000000000 +0000
@@ -83,7 +83,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2025-03-10"
+    DATE(submission_timestamp) >= "2025-03-11"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_last_seen_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_last_seen_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_last_seen_v1/query.sql	2025-03-11 11:53:35.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_last_seen_v1/query.sql	2025-03-11 12:16:09.000000000 +0000
@@ -4,11 +4,12 @@
     CAST(NULL AS INT64) AS days_seen_bits,
     CAST(NULL AS INT64) AS days_active_bits,
     CAST(NULL AS INT64) AS days_created_profile_bits,
+    isp,
   -- We make sure to delay * until the end so that as new columns are added
   -- to the daily table we can add those columns in the same order to the end
   -- of this schema, which may be necessary for the daily join query between
   -- the two tables to validate.
-    *
+    * EXCEPT (isp),
   FROM
     `moz-fx-data-shared-prod.accounts_cirrus_derived.baseline_clients_daily_v1`
   WHERE
@@ -28,7 +29,8 @@
       `moz-fx-data-shared-prod.udf.days_since_created_profile_as_28_bits`(
         DATE_DIFF(submission_date, first_run_date, DAY)
       ) AS days_created_profile_bits,
-      * EXCEPT (submission_date)
+      isp,
+      * EXCEPT (submission_date, isp)
     FROM
       `moz-fx-data-shared-prod.accounts_cirrus_derived.baseline_clients_daily_v1`
     WHERE
@@ -41,7 +43,8 @@
       days_seen_bits,
       days_active_bits,
       days_created_profile_bits,
-      * EXCEPT (submission_date, days_seen_bits, days_active_bits, days_created_profile_bits),
+      isp,
+      * EXCEPT (submission_date, days_seen_bits, days_active_bits, days_created_profile_bits, isp),
     FROM
       `moz-fx-data-shared-prod.accounts_cirrus_derived.baseline_clients_last_seen_v1`
     WHERE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/event_monitoring_live_v1/materialized_view.sql	2025-03-11 11:53:35.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/event_monitoring_live_v1/materialized_view.sql	2025-03-11 11:57:45.000000000 +0000
@@ -83,7 +83,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2025-03-10"
+    DATE(submission_timestamp) >= "2025-03-11"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/event_monitoring_live_v1/materialized_view.sql	2025-03-11 11:53:35.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/event_monitoring_live_v1/materialized_view.sql	2025-03-11 11:57:45.000000000 +0000
@@ -83,7 +83,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2025-03-10"
+    DATE(submission_timestamp) >= "2025-03-11"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql	2025-03-11 11:53:35.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql	2025-03-11 11:57:45.000000000 +0000
@@ -135,7 +135,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2025-03-10"
+    DATE(submission_timestamp) >= "2025-03-11"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_last_seen_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_last_seen_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_last_seen_v1/query.sql	2025-03-11 11:53:35.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_last_seen_v1/query.sql	2025-03-11 12:15:58.000000000 +0000
@@ -4,11 +4,12 @@
     CAST(NULL AS INT64) AS days_seen_bits,
     CAST(NULL AS INT64) AS days_active_bits,
     CAST(NULL AS INT64) AS days_created_profile_bits,
+    isp,
   -- We make sure to delay * until the end so that as new columns are added
   -- to the daily table we can add those columns in the same order to the end
   -- of this schema, which may be necessary for the daily join query between
   -- the two tables to validate.
-    *
+    * EXCEPT (isp),
   FROM
     `moz-fx-data-shared-prod.burnham_derived.baseline_clients_daily_v1`
   WHERE
@@ -28,7 +29,8 @@
       `moz-fx-data-shared-prod.udf.days_since_created_profile_as_28_bits`(
         DATE_DIFF(submission_date, first_run_date, DAY)
       ) AS days_created_profile_bits,
-      * EXCEPT (submission_date)
+      isp,
+      * EXCEPT (submission_date, isp)
     FROM
       `moz-fx-data-shared-prod.burnham_derived.baseline_clients_daily_v1`
     WHERE
@@ -41,7 +43,8 @@
       days_seen_bits,
       days_active_bits,
       days_created_profile_bits,
-      * EXCEPT (submission_date, days_seen_bits, days_active_bits, days_created_profile_bits),
+      isp,
+      * EXCEPT (submission_date, days_seen_bits, days_active_bits, days_created_profile_bits, isp),
     FROM
       `moz-fx-data-shared-prod.burnham_derived.baseline_clients_last_seen_v1`
     WHERE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/event_monitoring_live_v1/materialized_view.sql	2025-03-11 11:53:35.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/event_monitoring_live_v1/materialized_view.sql	2025-03-11 11:57:45.000000000 +0000
@@ -83,7 +83,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2025-03-10"
+    DATE(submission_timestamp) >= "2025-03-11"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml	2025-03-11 11:52:51.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml	2025-03-11 12:04:49.000000000 +0000
@@ -1,49 +1,49 @@
 fields:
-- mode: NULLABLE
-  name: submission_date
+- name: submission_date
   type: DATE
-- mode: NULLABLE
-  name: source
+  mode: NULLABLE
+- name: source
   type: STRING
-- mode: NULLABLE
-  name: event_type
+  mode: NULLABLE
+- name: event_type
   type: STRING
-- mode: NULLABLE
-  name: form_factor
+  mode: NULLABLE
+- name: form_factor
   type: STRING
-- mode: NULLABLE
-  name: country
+  mode: NULLABLE
+- name: country
   type: STRING
-- mode: NULLABLE
-  name: subdivision1
+  mode: NULLABLE
+- name: subdivision1
   type: STRING
-- mode: NULLABLE
-  name: advertiser
+  mode: NULLABLE
+- name: advertiser
   type: STRING
-- mode: NULLABLE
-  name: release_channel
+  mode: NULLABLE
+- name: release_channel
   type: STRING
-- mode: NULLABLE
-  name: position
+  mode: NULLABLE
+- name: position
   type: INTEGER
-- mode: NULLABLE
-  name: provider
+  mode: NULLABLE
+- name: provider
   type: STRING
-- mode: NULLABLE
-  name: match_type
+  mode: NULLABLE
+- name: match_type
   type: STRING
-- mode: NULLABLE
-  name: normalized_os
+  mode: NULLABLE
+- name: normalized_os
   type: STRING
-- mode: NULLABLE
-  name: suggest_data_sharing_enabled
+  mode: NULLABLE
+- name: suggest_data_sharing_enabled
   type: BOOLEAN
-- mode: NULLABLE
-  name: event_count
+  mode: NULLABLE
+- name: event_count
   type: INTEGER
-- mode: NULLABLE
-  name: user_count
+  mode: NULLABLE
+- name: user_count
   type: INTEGER
-- mode: NULLABLE
-  name: query_type
+  mode: NULLABLE
+- name: query_type
   type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml	2025-03-11 11:52:51.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml	2025-03-11 12:04:55.000000000 +0000
@@ -1,40 +1,40 @@
 fields:
-- mode: NULLABLE
-  name: submission_date
+- name: submission_date
   type: DATE
-- mode: NULLABLE
-  name: form_factor
+  mode: NULLABLE
+- name: form_factor
   type: STRING
-- mode: NULLABLE
-  name: country
+  mode: NULLABLE
+- name: country
   type: STRING
-- mode: NULLABLE
-  name: advertiser
+  mode: NULLABLE
+- name: advertiser
   type: STRING
-- mode: NULLABLE
-  name: normalized_os
+  mode: NULLABLE
+- name: normalized_os
   type: STRING
-- mode: NULLABLE
-  name: release_channel
+  mode: NULLABLE
+- name: release_channel
   type: STRING
-- mode: NULLABLE
-  name: position
+  mode: NULLABLE
+- name: position
   type: INTEGER
-- mode: NULLABLE
-  name: provider
+  mode: NULLABLE
+- name: provider
   type: STRING
-- mode: NULLABLE
-  name: match_type
+  mode: NULLABLE
+- name: match_type
   type: STRING
-- mode: NULLABLE
-  name: suggest_data_sharing_enabled
+  mode: NULLABLE
+- name: suggest_data_sharing_enabled
   type: BOOLEAN
-- mode: NULLABLE
-  name: impression_count
+  mode: NULLABLE
+- name: impression_count
   type: INTEGER
-- mode: NULLABLE
-  name: click_count
+  mode: NULLABLE
+- name: click_count
   type: INTEGER
-- mode: NULLABLE
-  name: query_type
+  mode: NULLABLE
+- name: query_type
   type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/event_monitoring_live_v1/materialized_view.sql	2025-03-11 11:53:35.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/event_monitoring_live_v1/materialized_view.sql	2025-03-11 11:57:45.000000000 +0000
@@ -83,7 +83,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2025-03-10"
+    DATE(submission_timestamp) >= "2025-03-11"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml	2025-03-11 11:52:51.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml	2025-03-11 12:04:23.000000000 +0000
@@ -26,6 +26,9 @@
 - name: adjust_network
   type: STRING
   mode: NULLABLE
+- name: install_source
+  type: STRING
+  mode: NULLABLE
 - name: retained_week_2
   type: BOOLEAN
   mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml	2025-03-11 11:52:51.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml	2025-03-11 12:04:30.000000000 +0000
@@ -48,6 +48,10 @@
   description: 'The type of source of a client installation.
 
     '
+- name: install_source
+  type: STRING
+  mode: NULLABLE
+  description: null
 - name: new_profiles
   type: INTEGER
   mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/metrics/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/metrics/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/metrics/schema.yaml	2025-03-11 11:53:35.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/metrics/schema.yaml	2025-03-11 12:09:28.000000000 +0000
@@ -27918,6 +27918,96 @@
           - name: value
             type: INTEGER
             mode: NULLABLE
+    - name: firefox_ai_runtime_engine_creation_success
+      type: RECORD
+      mode: REPEATED
+      fields:
+      - name: key
+        type: STRING
+        mode: NULLABLE
+      - name: value
+        type: RECORD
+        mode: NULLABLE
+        fields:
+        - name: bucket_count
+          type: INTEGER
+          mode: NULLABLE
+        - name: count
+          type: INTEGER
+          mode: NULLABLE
+        - name: histogram_type
+          type: STRING
+          mode: NULLABLE
+        - name: overflow
+          type: INTEGER
+          mode: NULLABLE
+        - name: range
+          type: FLOAT
+          mode: REPEATED
+        - name: sum
+          type: INTEGER
+          mode: NULLABLE
+        - name: time_unit
+          type: STRING
+          mode: NULLABLE
+        - name: underflow
+          type: INTEGER
+          mode: NULLABLE
+        - name: values
+          type: RECORD
+          mode: REPEATED
+          fields:
+          - name: key
+            type: STRING
+            mode: NULLABLE
+          - name: value
+            type: INTEGER
+            mode: NULLABLE
+    - name: firefox_ai_runtime_run_inference_success
+      type: RECORD
+      mode: REPEATED
+      fields:
+      - name: key
+        type: STRING
+        mode: NULLABLE
+      - name: value
+        type: RECORD
+        mode: NULLABLE
+        fields:
+        - name: bucket_count
+          type: INTEGER
+          mode: NULLABLE
+        - name: count
+          type: INTEGER
+          mode: NULLABLE
+        - name: histogram_type
+          type: STRING
+          mode: NULLABLE
+        - name: overflow
+          type: INTEGER
+          mode: NULLABLE
+        - name: range
+          type: FLOAT
+          mode: REPEATED
+        - name: sum
+          type: INTEGER
+          mode: NULLABLE
+        - name: time_unit
+          type: STRING
+          mode: NULLABLE
+        - name: underflow
+          type: INTEGER
+          mode: NULLABLE
+        - name: values
+          type: RECORD
+          mode: REPEATED
+          fields:
+          - name: key
+            type: STRING
+            mode: NULLABLE
+          - name: value
+            type: INTEGER
+            mode: NULLABLE
   - name: labeled_custom_distribution
     type: RECORD
     mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/metrics/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/metrics/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/metrics/view.sql	2025-03-11 11:53:35.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/metrics/view.sql	2025-03-11 12:01:35.000000000 +0000
@@ -1590,7 +1590,9 @@
       metrics.labeled_timing_distribution.workers_sync_worker_operation,
       metrics.labeled_timing_distribution.devtools_cold_toolbox_open_delay,
       metrics.labeled_timing_distribution.devtools_toolbox_page_reload_delay,
-      metrics.labeled_timing_distribution.devtools_warm_toolbox_open_delay
+      metrics.labeled_timing_distribution.devtools_warm_toolbox_open_delay,
+      metrics.labeled_timing_distribution.firefox_ai_runtime_engine_creation_success,
+      metrics.labeled_timing_distribution.firefox_ai_runtime_run_inference_success
     ) AS `labeled_timing_distribution`,
     STRUCT(
       metrics.labeled_custom_distribution.networking_http_3_ecn_ce_ect0_ratio,
@@ -3238,7 +3240,9 @@
       metrics.labeled_timing_distribution.workers_sync_worker_operation,
       metrics.labeled_timing_distribution.devtools_cold_toolbox_open_delay,
       metrics.labeled_timing_distribution.devtools_toolbox_page_reload_delay,
-      metrics.labeled_timing_distribution.devtools_warm_toolbox_open_delay
+      metrics.labeled_timing_distribution.devtools_warm_toolbox_open_delay,
+      metrics.labeled_timing_distribution.firefox_ai_runtime_engine_creation_success,
+      metrics.labeled_timing_distribution.firefox_ai_runtime_run_inference_success
     ) AS `labeled_timing_distribution`,
     STRUCT(
       metrics.labeled_custom_distribution.networking_http_3_ecn_ce_ect0_ratio,
@@ -4904,7 +4908,9 @@
       metrics.labeled_timing_distribution.workers_sync_worker_operation,
       metrics.labeled_timing_distribution.devtools_cold_toolbox_open_delay,
       metrics.labeled_timing_distribution.devtools_toolbox_page_reload_delay,
-      metrics.labeled_timing_distribution.devtools_warm_toolbox_open_delay
+      metrics.labeled_timing_distribution.devtools_warm_toolbox_open_delay,
+      metrics.labeled_timing_distribution.firefox_ai_runtime_engine_creation_success,
+      metrics.labeled_timing_distribution.firefox_ai_runtime_run_inference_success
     ) AS `labeled_timing_distribution`,
     STRUCT(
       metrics.labeled_custom_distribution.networking_http_3_ecn_ce_ect0_ratio,
@@ -6579,7 +6585,9 @@
       metrics.labeled_timing_distribution.workers_sync_worker_operation,
       metrics.labeled_timing_distribution.devtools_cold_toolbox_open_delay,
       metrics.labeled_timing_distribution.devtools_toolbox_page_reload_delay,
-      metrics.labeled_timing_distribution.devtools_warm_toolbox_open_delay
+      metrics.labeled_timing_distribution.devtools_warm_toolbox_open_delay,
+      metrics.labeled_timing_distribution.firefox_ai_runtime_engine_creation_success,
+      metrics.labeled_timing_distribution.firefox_ai_runtime_run_inference_success
     ) AS `labeled_timing_distribution`,
     STRUCT(
       metrics.labeled_custom_distribution.networking_http_3_ecn_ce_ect0_ratio,
@@ -8236,7 +8244,9 @@
       metrics.labeled_timing_distribution.workers_sync_worker_operation,
       metrics.labeled_timing_distribution.devtools_cold_toolbox_open_delay,
       metrics.labeled_timing_distribution.devtools_toolbox_page_reload_delay,
-      metrics.labeled_timing_distribution.devtools_warm_toolbox_open_delay
+      metrics.labeled_timing_distribution.devtools_warm_toolbox_open_delay,
+      metrics.labeled_timing_distribution.firefox_ai_runtime_engine_creation_success,
+      metrics.labeled_timing_distribution.firefox_ai_runtime_run_inference_success
     ) AS `labeled_timing_distribution`,
     STRUCT(
       metrics.labeled_custom_distribution.networking_http_3_ecn_ce_ect0_ratio,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_crashreporter_derived/baseline_clients_last_seen_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_crashreporter_derived/baseline_clients_last_seen_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_crashreporter_derived/baseline_clients_last_seen_v1/query.sql	2025-03-11 11:53:34.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_crashreporter_derived/baseline_clients_last_seen_v1/query.sql	2025-03-11 12:16:14.000000000 +0000
@@ -4,11 +4,12 @@
     CAST(NULL AS INT64) AS days_seen_bits,
     CAST(NULL AS INT64) AS days_active_bits,
     CAST(NULL AS INT64) AS days_created_profile_bits,
+    isp,
   -- We make sure to delay * until the end so that as new columns are added
   -- to the daily table we can add those columns in the same order to the end
   -- of this schema, which may be necessary for the daily join query between
   -- the two tables to validate.
-    *
+    * EXCEPT (isp),
   FROM
     `moz-fx-data-shared-prod.firefox_crashreporter_derived.baseline_clients_daily_v1`
   WHERE
@@ -28,7 +29,8 @@
       `moz-fx-data-shared-prod.udf.days_since_created_profile_as_28_bits`(
         DATE_DIFF(submission_date, first_run_date, DAY)
       ) AS days_created_profile_bits,
-      * EXCEPT (submission_date)
+      isp,
+      * EXCEPT (submission_date, isp)
     FROM
       `moz-fx-data-shared-prod.firefox_crashreporter_derived.baseline_clients_daily_v1`
     WHERE
@@ -41,7 +43,8 @@
       days_seen_bits,
       days_active_bits,
       days_created_profile_bits,
-      * EXCEPT (submission_date, days_seen_bits, days_active_bits, days_created_profile_bits),
+      isp,
+      * EXCEPT (submission_date, days_seen_bits, days_active_bits, days_created_profile_bits, isp),
     FROM
       `moz-fx-data-shared-prod.firefox_crashreporter_derived.baseline_clients_last_seen_v1`
     WHERE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_crashreporter_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_crashreporter_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_crashreporter_derived/event_monitoring_live_v1/materialized_view.sql	2025-03-11 11:53:34.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_crashreporter_derived/event_monitoring_live_v1/materialized_view.sql	2025-03-11 11:57:46.000000000 +0000
@@ -83,7 +83,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2025-03-10"
+    DATE(submission_timestamp) >= "2025-03-11"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/ltv_states/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/ltv_states/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/ltv_states/schema.yaml	2025-03-11 11:52:51.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/ltv_states/schema.yaml	2025-03-11 12:04:53.000000000 +0000
@@ -1,64 +1,66 @@
 fields:
-- description: Unique ID for the client installation.
-  mode: NULLABLE
-  name: client_id
+- name: client_id
   type: STRING
-- description: Sample ID - A number ranging from 0 - 99 based on client ID; used to pull a small sample of data related to a subset of clients over time
   mode: NULLABLE
-  name: sample_id
-  type: INT64
-- description: Submission Date
+  description: Unique ID for the client installation.
+- name: sample_id
+  type: INTEGER
   mode: NULLABLE
-  name: submission_date
+  description: Sample ID - A number ranging from 0 - 99 based on client ID; used to
+    pull a small sample of data related to a subset of clients over time
+- name: submission_date
   type: DATE
-- description: First Seen Date - The date this client was first seen
   mode: NULLABLE
-  name: first_seen_date
+  description: Submission Date
+- name: first_seen_date
   type: DATE
-- description: Days Since First Seen - The number of days since the client was first seen
   mode: NULLABLE
-  name: days_since_first_seen
-  type: INT64
-- description: Days Since Active
+  description: First Seen Date - The date this client was first seen
+- name: days_since_first_seen
+  type: INTEGER
   mode: NULLABLE
-  name: days_since_active
-  type: INT64
-- description: First Reported Country - The country this client ID was first reported from
+  description: Days Since First Seen - The number of days since the client was first
+    seen
+- name: days_since_active
+  type: INTEGER
   mode: NULLABLE
-  name: first_reported_country
+  description: Days Since Active
+- name: first_reported_country
   type: STRING
-- description: Attribution
   mode: NULLABLE
-  name: attribution
+  description: First Reported Country - The country this client ID was first reported
+    from
+- name: attribution
   type: RECORD
+  mode: NULLABLE
   fields:
-  - mode: NULLABLE
-    name: source
+  - name: source
     type: STRING
+    mode: NULLABLE
     description: Attribution Source
-  - mode: NULLABLE
-    name: medium
+  - name: medium
     type: STRING
+    mode: NULLABLE
     description: Attribution Medium
-  - mode: NULLABLE
-    name: campaign
+  - name: campaign
     type: STRING
+    mode: NULLABLE
     description: Attribution Campaign
-  - mode: NULLABLE
-    name: content
+  - name: content
     type: STRING
+    mode: NULLABLE
     description: Attribution Content
-  - mode: NULLABLE
-    name: experiment
+  - name: experiment
     type: STRING
+    mode: NULLABLE
     description: Attribution Experiment
-  - mode: NULLABLE
-    name: variation
+  - name: variation
     type: STRING
+    mode: NULLABLE
     description: Attribution Variation
-  - mode: NULLABLE
-    name: dltoken
+  - name: dltoken
     type: STRING
+    mode: NULLABLE
     description: Attribution Download Token
   - name: dlsource
     type: STRING
@@ -68,40 +70,43 @@
     type: STRING
     mode: NULLABLE
     description: Attribution UA
-- description: Active
-  mode: NULLABLE
-  name: active
-  type: INT64
-- description: Ad Clicks - The number of ad clicks from this client on the submission date
+  description: Attribution
+- name: active
+  type: INTEGER
   mode: NULLABLE
-  name: ad_clicks
-  type: INT64
-- description: Total Historic Ad Clicks - The number of ad clicks from this client on or before the submission date
+  description: Active
+- name: ad_clicks
+  type: INTEGER
   mode: NULLABLE
-  name: total_historic_ad_clicks
-  type: INT64
-- description: Days Seen Bytes
+  description: Ad Clicks - The number of ad clicks from this client on the submission
+    date
+- name: total_historic_ad_clicks
+  type: INTEGER
   mode: NULLABLE
-  name: days_seen_bytes
+  description: Total Historic Ad Clicks - The number of ad clicks from this client
+    on or before the submission date
+- name: days_seen_bytes
   type: BYTES
-- description: Pattern
   mode: NULLABLE
-  name: pattern
+  description: Days Seen Bytes
+- name: pattern
   type: INTEGER
-- description: Death Time
   mode: NULLABLE
-  name: death_time
+  description: Pattern
+- name: death_time
   type: INTEGER
-- description: Max Days
   mode: NULLABLE
-  name: max_days
+  description: Death Time
+- name: max_days
   type: INTEGER
-- description: Markov States
   mode: NULLABLE
-  name: markov_states
+  description: Max Days
+- name: markov_states
   type: RECORD
-  fields:
-  - description: Desktop States V1
     mode: NULLABLE
-    name: desktop_states_v1
+  fields:
+  - name: desktop_states_v1
     type: STRING
+    mode: NULLABLE
+    description: Desktop States V1
+  description: Markov States
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/metrics/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/metrics/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/metrics/schema.yaml	2025-03-11 11:53:35.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/metrics/schema.yaml	2025-03-11 11:55:33.000000000 +0000
@@ -41200,6 +41200,106 @@
         histogram DEVTOOLS_WARM_TOOLBOX_OPEN_DELAY_MS.
 
         '
+    - name: firefox_ai_runtime_engine_creation_success
+      type: RECORD
+      mode: REPEATED
+      fields:
+      - name: key
+        type: STRING
+        mode: NULLABLE
+      - name: value
+        type: RECORD
+        mode: NULLABLE
+        fields:
+        - name: bucket_count
+          type: INTEGER
+          mode: NULLABLE
+        - name: count
+          type: INTEGER
+          mode: NULLABLE
+          description: This was accidentally sent in the past and is now deprecated.
+            See https://bugzilla.mozilla.org/show_bug.cgi?id=1799509#c5
+        - name: histogram_type
+          type: STRING
+          mode: NULLABLE
+        - name: overflow
+          type: INTEGER
+          mode: NULLABLE
+        - name: range
+          type: FLOAT
+          mode: REPEATED
+        - name: sum
+          type: INTEGER
+          mode: NULLABLE
+        - name: time_unit
+          type: STRING
+          mode: NULLABLE
+        - name: underflow
+          type: INTEGER
+          mode: NULLABLE
+        - name: values
+          type: RECORD
+          mode: REPEATED
+          fields:
+          - name: key
+            type: STRING
+            mode: NULLABLE
+          - name: value
+            type: INTEGER
+            mode: NULLABLE
+      description: 'An inference engine was created
+
+        '
+    - name: firefox_ai_runtime_run_inference_success
+      type: RECORD
+      mode: REPEATED
+      fields:
+      - name: key
+        type: STRING
+        mode: NULLABLE
+      - name: value
+        type: RECORD
+        mode: NULLABLE
+        fields:
+        - name: bucket_count
+          type: INTEGER
+          mode: NULLABLE
+        - name: count
+          type: INTEGER
+          mode: NULLABLE
+          description: This was accidentally sent in the past and is now deprecated.
+            See https://bugzilla.mozilla.org/show_bug.cgi?id=1799509#c5
+        - name: histogram_type
+          type: STRING
+          mode: NULLABLE
+        - name: overflow
+          type: INTEGER
+          mode: NULLABLE
+        - name: range
+          type: FLOAT
+          mode: REPEATED
+        - name: sum
+          type: INTEGER
+          mode: NULLABLE
+        - name: time_unit
+          type: STRING
+          mode: NULLABLE
+        - name: underflow
+          type: INTEGER
+          mode: NULLABLE
+        - name: values
+          type: RECORD
+          mode: REPEATED
+          fields:
+          - name: key
+            type: STRING
+            mode: NULLABLE
+          - name: value
+            type: INTEGER
+            mode: NULLABLE
+      description: 'An inference engine was created
+
+        '
   - name: labeled_custom_distribution
     type: RECORD
     mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/newtab_live/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/newtab_live/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/newtab_live/schema.yaml	2025-03-11 11:52:51.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop/newtab_live/schema.yaml	2025-03-11 12:05:01.000000000 +0000
@@ -1,48 +1,47 @@
 fields:
-- description: Submission Timestamp
-  mode: NULLABLE
-  name: submission_timestamp
+- name: submission_timestamp
   type: TIMESTAMP
-- description: Normalized Country Code, Examples - US, AR, BR, etc.
   mode: NULLABLE
-  name: normalized_country_code
+  description: Submission Timestamp
+- name: normalized_country_code
   type: STRING
-- description: Normalized Channel, Examples - release, nightly, aurora, esr, beta
   mode: NULLABLE
-  name: normalized_channel
+  description: Normalized Country Code, Examples - US, AR, BR, etc.
+- name: normalized_channel
   type: STRING
-- description: Document ID
   mode: NULLABLE
-  name: document_id
+  description: Normalized Channel, Examples - release, nightly, aurora, esr, beta
+- name: document_id
   type: STRING
-- description: Pocket Enabled
   mode: NULLABLE
-  name: pocket_enabled
+  description: Document ID
+- name: pocket_enabled
   type: BOOLEAN
-- description: Pocket Sponsored Stories Enabled
   mode: NULLABLE
-  name: pocket_sponsored_stories_enabled
+  description: Pocket Enabled
+- name: pocket_sponsored_stories_enabled
   type: BOOLEAN
-- description: Newtab Locale
   mode: NULLABLE
-  name: newtab_locale
+  description: Pocket Sponsored Stories Enabled
+- name: newtab_locale
   type: STRING
-- description: App Build
   mode: NULLABLE
-  name: app_build
+  description: Newtab Locale
+- name: app_build
   type: STRING
-- description: App Display Version
   mode: NULLABLE
-  name: app_display_version
+  description: App Build
+- name: app_display_version
   type: STRING
-- description: Client ID
   mode: NULLABLE
-  name: client_id
+  description: App Display Version
+- name: client_id
   type: STRING
+  mode: NULLABLE
+  description: Client ID
 - name: events
   type: RECORD
   mode: REPEATED
-  description: Events
   fields:
   - name: category
     type: STRING
@@ -51,7 +50,6 @@
   - name: extra
     type: RECORD
     mode: REPEATED
-    description: Extras
     fields:
     - name: key
       type: STRING
@@ -61,6 +59,7 @@
       type: STRING
       mode: NULLABLE
       description: Value
+    description: Extras
   - name: name
     type: STRING
     mode: NULLABLE
@@ -69,3 +68,4 @@
     type: INTEGER
     mode: NULLABLE
     description: Event Timestamp
+  description: Events
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_defaultagent_derived/baseline_clients_last_seen_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_defaultagent_derived/baseline_clients_last_seen_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_defaultagent_derived/baseline_clients_last_seen_v1/query.sql	2025-03-11 11:53:35.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_defaultagent_derived/baseline_clients_last_seen_v1/query.sql	2025-03-11 12:15:55.000000000 +0000
@@ -4,11 +4,12 @@
     CAST(NULL AS INT64) AS days_seen_bits,
     CAST(NULL AS INT64) AS days_active_bits,
     CAST(NULL AS INT64) AS days_created_profile_bits,
+    isp,
   -- We make sure to delay * until the end so that as new columns are added
   -- to the daily table we can add those columns in the same order to the end
   -- of this schema, which may be necessary for the daily join query between
   -- the two tables to validate.
-    *
+    * EXCEPT (isp),
   FROM
     `moz-fx-data-shared-prod.firefox_desktop_background_defaultagent_derived.baseline_clients_daily_v1`
   WHERE
@@ -28,7 +29,8 @@
       `moz-fx-data-shared-prod.udf.days_since_created_profile_as_28_bits`(
         DATE_DIFF(submission_date, first_run_date, DAY)
       ) AS days_created_profile_bits,
-      * EXCEPT (submission_date)
+      isp,
+      * EXCEPT (submission_date, isp)
     FROM
       `moz-fx-data-shared-prod.firefox_desktop_background_defaultagent_derived.baseline_clients_daily_v1`
     WHERE
@@ -41,7 +43,8 @@
       days_seen_bits,
       days_active_bits,
       days_created_profile_bits,
-      * EXCEPT (submission_date, days_seen_bits, days_active_bits, days_created_profile_bits),
+      isp,
+      * EXCEPT (submission_date, days_seen_bits, days_active_bits, days_created_profile_bits, isp),
     FROM
       `moz-fx-data-shared-prod.firefox_desktop_background_defaultagent_derived.baseline_clients_last_seen_v1`
     WHERE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_defaultagent_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_defaultagent_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_defaultagent_derived/event_monitoring_live_v1/materialized_view.sql	2025-03-11 11:53:35.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_defaultagent_derived/event_monitoring_live_v1/materialized_view.sql	2025-03-11 11:57:46.000000000 +0000
@@ -83,7 +83,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2025-03-10"
+    DATE(submission_timestamp) >= "2025-03-11"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/baseline_clients_last_seen_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/baseline_clients_last_seen_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/baseline_clients_last_seen_v1/query.sql	2025-03-11 11:53:34.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/baseline_clients_last_seen_v1/query.sql	2025-03-11 12:16:14.000000000 +0000
@@ -4,11 +4,12 @@
     CAST(NULL AS INT64) AS days_seen_bits,
     CAST(NULL AS INT64) AS days_active_bits,
     CAST(NULL AS INT64) AS days_created_profile_bits,
+    isp,
   -- We make sure to delay * until the end so that as new columns are added
   -- to the daily table we can add those columns in the same order to the end
   -- of this schema, which may be necessary for the daily join query between
   -- the two tables to validate.
-    *
+    * EXCEPT (isp),
   FROM
     `moz-fx-data-shared-prod.firefox_desktop_background_tasks_derived.baseline_clients_daily_v1`
   WHERE
@@ -28,7 +29,8 @@
       `moz-fx-data-shared-prod.udf.days_since_created_profile_as_28_bits`(
         DATE_DIFF(submission_date, first_run_date, DAY)
       ) AS days_created_profile_bits,
-      * EXCEPT (submission_date)
+      isp,
+      * EXCEPT (submission_date, isp)
     FROM
       `moz-fx-data-shared-prod.firefox_desktop_background_tasks_derived.baseline_clients_daily_v1`
     WHERE
@@ -41,7 +43,8 @@
       days_seen_bits,
       days_active_bits,
       days_created_profile_bits,
-      * EXCEPT (submission_date, days_seen_bits, days_active_bits, days_created_profile_bits),
+      isp,
+      * EXCEPT (submission_date, days_seen_bits, days_active_bits, days_created_profile_bits, isp),
     FROM
       `moz-fx-data-shared-prod.firefox_desktop_background_tasks_derived.baseline_clients_last_seen_v1`
     WHERE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/event_monitoring_live_v1/materialized_view.sql	2025-03-11 11:53:34.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/event_monitoring_live_v1/materialized_view.sql	2025-03-11 11:57:46.000000000 +0000
@@ -109,7 +109,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2025-03-10"
+    DATE(submission_timestamp) >= "2025-03-11"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update/metrics/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update/metrics/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update/metrics/schema.yaml	2025-03-11 11:53:35.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update/metrics/schema.yaml	2025-03-11 11:55:27.000000000 +0000
@@ -33225,6 +33225,106 @@
         histogram DEVTOOLS_WARM_TOOLBOX_OPEN_DELAY_MS.
 
         '
+    - name: firefox_ai_runtime_engine_creation_success
+      type: RECORD
+      mode: REPEATED
+      fields:
+      - name: key
+        type: STRING
+        mode: NULLABLE
+      - name: value
+        type: RECORD
+        mode: NULLABLE
+        fields:
+        - name: bucket_count
+          type: INTEGER
+          mode: NULLABLE
+        - name: count
+          type: INTEGER
+          mode: NULLABLE
+          description: This was accidentally sent in the past and is now deprecated.
+            See https://bugzilla.mozilla.org/show_bug.cgi?id=1799509#c5
+        - name: histogram_type
+          type: STRING
+          mode: NULLABLE
+        - name: overflow
+          type: INTEGER
+          mode: NULLABLE
+        - name: range
+          type: FLOAT
+          mode: REPEATED
+        - name: sum
+          type: INTEGER
+          mode: NULLABLE
+        - name: time_unit
+          type: STRING
+          mode: NULLABLE
+        - name: underflow
+          type: INTEGER
+          mode: NULLABLE
+        - name: values
+          type: RECORD
+          mode: REPEATED
+          fields:
+          - name: key
+            type: STRING
+            mode: NULLABLE
+          - name: value
+            type: INTEGER
+            mode: NULLABLE
+      description: 'An inference engine was created
+
+        '
+    - name: firefox_ai_runtime_run_inference_success
+      type: RECORD
+      mode: REPEATED
+      fields:
+      - name: key
+        type: STRING
+        mode: NULLABLE
+      - name: value
+        type: RECORD
+        mode: NULLABLE
+        fields:
+        - name: bucket_count
+          type: INTEGER
+          mode: NULLABLE
+        - name: count
+          type: INTEGER
+          mode: NULLABLE
+          description: This was accidentally sent in the past and is now deprecated.
+            See https://bugzilla.mozilla.org/show_bug.cgi?id=1799509#c5
+        - name: histogram_type
+          type: STRING
+          mode: NULLABLE
+        - name: overflow
+          type: INTEGER
+          mode: NULLABLE
+        - name: range
+          type: FLOAT
+          mode: REPEATED
+        - name: sum
+          type: INTEGER
+          mode: NULLABLE
+        - name: time_unit
+          type: STRING
+          mode: NULLABLE
+        - name: underflow
+          type: INTEGER
+          mode: NULLABLE
+        - name: values
+          type: RECORD
+          mode: REPEATED
+          fields:
+          - name: key
+            type: STRING
+            mode: NULLABLE
+          - name: value
+            type: INTEGER
+            mode: NULLABLE
+      description: 'An inference engine was created
+
+        '
   - name: labeled_custom_distribution
     type: RECORD
     mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update_derived/baseline_clients_last_seen_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update_derived/baseline_clients_last_seen_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update_derived/baseline_clients_last_seen_v1/query.sql	2025-03-11 11:53:35.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update_derived/baseline_clients_last_seen_v1/query.sql	2025-03-11 12:16:14.000000000 +0000
@@ -4,11 +4,12 @@
     CAST(NULL AS INT64) AS days_seen_bits,
     CAST(NULL AS INT64) AS days_active_bits,
     CAST(NULL AS INT64) AS days_created_profile_bits,
+    isp,
   -- We make sure to delay * until the end so that as new columns are added
   -- to the daily table we can add those columns in the same order to the end
   -- of this schema, which may be necessary for the daily join query between
   -- the two tables to validate.
-    *
+    * EXCEPT (isp),
   FROM
     `moz-fx-data-shared-prod.firefox_desktop_background_update_derived.baseline_clients_daily_v1`
   WHERE
@@ -28,7 +29,8 @@
       `moz-fx-data-shared-prod.udf.days_since_created_profile_as_28_bits`(
         DATE_DIFF(submission_date, first_run_date, DAY)
       ) AS days_created_profile_bits,
-      * EXCEPT (submission_date)
+      isp,
+      * EXCEPT (submission_date, isp)
     FROM
       `moz-fx-data-shared-prod.firefox_desktop_background_update_derived.baseline_clients_daily_v1`
     WHERE
@@ -41,7 +43,8 @@
       days_seen_bits,
       days_active_bits,
       days_created_profile_bits,
-      * EXCEPT (submission_date, days_seen_bits, days_active_bits, days_created_profile_bits),
+      isp,
+      * EXCEPT (submission_date, days_seen_bits, days_active_bits, days_created_profile_bits, isp),
     FROM
       `moz-fx-data-shared-prod.firefox_desktop_background_update_derived.baseline_clients_last_seen_v1`
     WHERE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update_derived/event_monitoring_live_v1/materialized_view.sql	2025-03-11 11:53:35.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update_derived/event_monitoring_live_v1/materialized_view.sql	2025-03-11 11:57:46.000000000 +0000
@@ -83,7 +83,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2025-03-10"
+    DATE(submission_timestamp) >= "2025-03-11"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/baseline_clients_last_seen_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/baseline_clients_last_seen_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/baseline_clients_last_seen_v1/query.sql	2025-03-11 11:53:35.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/baseline_clients_last_seen_v1/query.sql	2025-03-11 12:16:14.000000000 +0000
@@ -5,11 +5,12 @@
     CAST(NULL AS INT64) AS days_active_bits,
     CAST(NULL AS INT64) AS days_created_profile_bits,
     CAST(NULL AS INT64) AS days_desktop_active_bits,
+    isp,
   -- We make sure to delay * until the end so that as new columns are added
   -- to the daily table we can add those columns in the same order to the end
   -- of this schema, which may be necessary for the daily join query between
   -- the two tables to validate.
-    *
+    * EXCEPT (isp),
   FROM
     `moz-fx-data-shared-prod.firefox_desktop_derived.baseline_clients_daily_v1`
   WHERE
@@ -32,7 +33,8 @@
       CAST(TRUE AS INT64) & CAST(browser_engagement_uri_count > 0 AS INT64) & CAST(
         browser_engagement_active_ticks > 0 AS INT64
       ) AS days_desktop_active_bits,
-      * EXCEPT (submission_date)
+      isp,
+      * EXCEPT (submission_date, isp)
     FROM
       `moz-fx-data-shared-prod.firefox_desktop_derived.baseline_clients_daily_v1`
     WHERE
@@ -46,12 +48,14 @@
       days_active_bits,
       days_desktop_active_bits,
       days_created_profile_bits,
+      isp,
       * EXCEPT (
         submission_date,
         days_seen_bits,
         days_active_bits,
         days_desktop_active_bits,
-        days_created_profile_bits
+        days_created_profile_bits,
+        isp
       ),
     FROM
       `moz-fx-data-shared-prod.firefox_desktop_derived.baseline_clients_last_seen_v1`
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/event_monitoring_live_v1/materialized_view.sql	2025-03-11 11:53:35.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/event_monitoring_live_v1/materialized_view.sql	2025-03-11 11:57:46.000000000 +0000
@@ -187,7 +187,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2025-03-10"
+    DATE(submission_timestamp) >= "2025-03-11"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_translations_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_translations_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_translations_derived/event_monitoring_live_v1/materialized_view.sql	2025-03-11 11:53:35.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_translations_derived/event_monitoring_live_v1/materialized_view.sql	2025-03-11 11:57:46.000000000 +0000
@@ -109,7 +109,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2025-03-10"
+    DATE(submission_timestamp) >= "2025-03-11"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/metrics/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/metrics/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/metrics/schema.yaml	2025-03-11 11:53:34.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/metrics/schema.yaml	2025-03-11 12:12:33.000000000 +0000
@@ -25152,6 +25152,96 @@
           - name: value
             type: INTEGER
             mode: NULLABLE
+    - name: firefox_ai_runtime_engine_creation_success
+      type: RECORD
+      mode: REPEATED
+      fields:
+      - name: key
+        type: STRING
+        mode: NULLABLE
+      - name: value
+        type: RECORD
+        mode: NULLABLE
+        fields:
+        - name: bucket_count
+          type: INTEGER
+          mode: NULLABLE
+        - name: count
+          type: INTEGER
+          mode: NULLABLE
+        - name: histogram_type
+          type: STRING
+          mode: NULLABLE
+        - name: overflow
+          type: INTEGER
+          mode: NULLABLE
+        - name: range
+          type: FLOAT
+          mode: REPEATED
+        - name: sum
+          type: INTEGER
+          mode: NULLABLE
+        - name: time_unit
+          type: STRING
+          mode: NULLABLE
+        - name: underflow
+          type: INTEGER
+          mode: NULLABLE
+        - name: values
+          type: RECORD
+          mode: REPEATED
+          fields:
+          - name: key
+            type: STRING
+            mode: NULLABLE
+          - name: value
+            type: INTEGER
+            mode: NULLABLE
+    - name: firefox_ai_runtime_run_inference_success
+      type: RECORD
+      mode: REPEATED
+      fields:
+      - name: key
+        type: STRING
+        mode: NULLABLE
+      - name: value
+        type: RECORD
+        mode: NULLABLE
+        fields:
+        - name: bucket_count
+          type: INTEGER
+          mode: NULLABLE
+        - name: count
+          type: INTEGER
+          mode: NULLABLE
+        - name: histogram_type
+          type: STRING
+          mode: NULLABLE
+        - name: overflow
+          type: INTEGER
+          mode: NULLABLE
+        - name: range
+          type: FLOAT
+          mode: REPEATED
+        - name: sum
+          type: INTEGER
+          mode: NULLABLE
+        - name: time_unit
+          type: STRING
+          mode: NULLABLE
+        - name: underflow
+          type: INTEGER
+          mode: NULLABLE
+        - name: values
+          type: RECORD
+          mode: REPEATED
+          fields:
+          - name: key
+            type: STRING
+            mode: NULLABLE
+          - name: value
+            type: INTEGER
+            mode: NULLABLE
   - name: labeled_custom_distribution
     type: RECORD
     mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/metrics/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/metrics/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/metrics/view.sql	2025-03-11 11:53:34.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/metrics/view.sql	2025-03-11 11:59:39.000000000 +0000
@@ -1367,7 +1367,9 @@
       metrics.labeled_timing_distribution.workers_sync_worker_operation,
       metrics.labeled_timing_distribution.devtools_cold_toolbox_open_delay,
       metrics.labeled_timing_distribution.devtools_toolbox_page_reload_delay,
-      metrics.labeled_timing_distribution.devtools_warm_toolbox_open_delay
+      metrics.labeled_timing_distribution.devtools_warm_toolbox_open_delay,
+      metrics.labeled_timing_distribution.firefox_ai_runtime_engine_creation_success,
+      metrics.labeled_timing_distribution.firefox_ai_runtime_run_inference_success
     ) AS `labeled_timing_distribution`,
     STRUCT(
       metrics.labeled_custom_distribution.networking_http_3_ecn_ce_ect0_ratio,
@@ -2791,7 +2793,9 @@
       metrics.labeled_timing_distribution.workers_sync_worker_operation,
       metrics.labeled_timing_distribution.devtools_cold_toolbox_open_delay,
       metrics.labeled_timing_distribution.devtools_toolbox_page_reload_delay,
-      metrics.labeled_timing_distribution.devtools_warm_toolbox_open_delay
+      metrics.labeled_timing_distribution.devtools_warm_toolbox_open_delay,
+      metrics.labeled_timing_distribution.firefox_ai_runtime_engine_creation_success,
+      metrics.labeled_timing_distribution.firefox_ai_runtime_run_inference_success
     ) AS `labeled_timing_distribution`,
     STRUCT(
       metrics.labeled_custom_distribution.networking_http_3_ecn_ce_ect0_ratio,
@@ -4215,7 +4219,9 @@
       metrics.labeled_timing_distribution.workers_sync_worker_operation,
       metrics.labeled_timing_distribution.devtools_cold_toolbox_open_delay,
       metrics.labeled_timing_distribution.devtools_toolbox_page_reload_delay,
-      metrics.labeled_timing_distribution.devtools_warm_toolbox_open_delay
+      metrics.labeled_timing_distribution.devtools_warm_toolbox_open_delay,
+      metrics.labeled_timing_distribution.firefox_ai_runtime_engine_creation_success,
+      metrics.labeled_timing_distribution.firefox_ai_runtime_run_inference_success
     ) AS `labeled_timing_distribution`,
     STRUCT(
       metrics.labeled_custom_distribution.networking_http_3_ecn_ce_ect0_ratio,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/use_counters/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/use_counters/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android/use_counters/schema.yaml	2025-03-11 11:53:34.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android/use_counters/schema.yaml	2025-03-11 12:09:02.000000000 +0000
@@ -1,11 +1,11 @@
 fields:
 - name: normalized_app_id
-  type: STRING
   mode: NULLABLE
+  type: STRING
   description: App ID of the channel data was received from
 - name: normalized_channel
-  type: STRING
   mode: NULLABLE
+  type: STRING
   description: Normalized channel name
 - name: additional_properties
   type: STRING
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/glean_dictionary_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/glean_dictionary_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/glean_dictionary_derived/event_monitoring_live_v1/materialized_view.sql	2025-03-11 11:53:35.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/glean_dictionary_derived/event_monitoring_live_v1/materialized_view.sql	2025-03-11 11:57:46.000000000 +0000
@@ -83,7 +83,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2025-03-10"
+    DATE(submission_timestamp) >= "2025-03-11"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/gleanjs_docs_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/gleanjs_docs_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/gleanjs_docs_derived/event_monitoring_live_v1/materialized_view.sql	2025-03-11 11:53:35.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/gleanjs_docs_derived/event_monitoring_live_v1/materialized_view.sql	2025-03-11 11:57:46.000000000 +0000
@@ -83,7 +83,7 @@
   LEFT JOIN
     UNNEST(event.extra) AS event_extra
   WHERE
-    DATE(submission_timestamp) >= "2025-03-10"
+    DATE(submission_timestamp) >= "2025-03-11"
   GROUP BY
     submission_date,
     window_start,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/google_search_console/limited_historical_search_impressions_by_page/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/google_search_console/limited_historical_search_impressions_by_page/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/google_search_console/limited_historical_search_impressions_by_page/schema.yaml	2025-03-11 11:52:51.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/google_search_console/limited_historical_search_impressions_by_page/schema.yaml	2025-03-11 12:04:17.000000000 +0000
@@ -6,9 +6,10 @@
 - name: site_url
   type: STRING
   mode: NULLABLE
-  description: |-
-    For domain properties, this will be `sc-domain:` followed by the domain name.
-    For URL-prefix properties, it will be the full URL of the property definition.
+  description: 'For domain properties, this will be `sc-domain:` followed by the domain
+    name.
+
+    For URL-prefix properties, it will be the full URL of the property definition.'
 - name: site_domain_name
   type: STRING
   mode: NULLABLE
@@ -28,19 +29,23 @@
 - name: localized_site_code
   type: STRING
   mode: NULLABLE
-  description: Localized site code such as `en-US` or `de` found in the first segment of the page URL path (if any).
+  description: Localized site code such as `en-US` or `de` found in the first segment
+    of the page URL path (if any).
 - name: localized_site
   type: STRING
   mode: NULLABLE
-  description: Description of the localized site language and/or country based on `localized_site_code` (if any).
+  description: Description of the localized site language and/or country based on
+    `localized_site_code` (if any).
 - name: localized_site_language_code
   type: STRING
   mode: NULLABLE
-  description: Localized site language code in ISO-639-alpha-2 format found in the first segment of the page URL path (if any).
+  description: Localized site language code in ISO-639-alpha-2 format found in the
+    first segment of the page URL path (if any).
 - name: localized_site_language
   type: STRING
   mode: NULLABLE
-  description: Localized site language based on `localized_site_language_code` (if any).
+  description: Localized site language based on `localized_site_language_code` (if
+    any).
 - name: query
   type: STRING
   mode: NULLABLE
@@ -48,20 +53,15 @@
 - name: query_type
   type: STRING
   mode: NULLABLE
-  description: |-
-    Type of search query:
-      * Brand: Query contained one or more Mozilla brand keywords.
-      * Non-Brand: Query didn't contain any Mozilla brand keywords.
-      * Unknown: Query couldn't be classified.
+  description: "Type of search query:\n  * Brand: Query contained one or more Mozilla\
+    \ brand keywords.\n  * Non-Brand: Query didn't contain any Mozilla brand keywords.\n\
+    \  * Unknown: Query couldn't be classified."
 - name: search_type
   type: STRING
   mode: NULLABLE
-  description: |-
-    Where the link was seen by the user:
-      * Web: In Google Search's default "All" tab.
-      * Image: In Google Search's "Images" tab.
-      * Video: In Google Search's "Videos" tab.
-      * News: In Google Search's "News" tab.
+  description: "Where the link was seen by the user:\n  * Web: In Google Search's\
+    \ default \"All\" tab.\n  * Image: In Google Search's \"Images\" tab.\n  * Video:\
+    \ In Google Search's \"Videos\" tab.\n  * News: In Google Search's \"News\" tab."
 - name: user_country_code
   type: STRING
   mode: NULLABLE
@@ -81,12 +81,13 @@
 - name: device_type
   type: STRING
   mode: NULLABLE
-  description: |-
-    The type of device on which the user was searching: Desktop, Mobile, or Tablet.
+  description: 'The type of device on which the user was searching: Desktop, Mobile,
+    or Tablet.'
 - name: impressions
   type: INTEGER
   mode: NULLABLE
-  description: The number of times that search results with a link to the page were shown to a user.
+  description: The number of times that search results with a link to the page were
+    shown to a user.
 - name: clicks
   type: INTEGER
   mode: NULLABLE
@@ -94,4 +95,5 @@
 - name: average_position
   type: FLOAT
   mode: NULLABLE
-  description: The average position of the page in the search results, where `1` is the topmost position.
+  description: The average position of the page in the search results, where `1` is
+    the topmost position.
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/google_search_console/limited_historical_search_impressions_by_site/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/google_search_console/limited_historical_search_impressions_by_site/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/go

⚠️ Only part of the diff is displayed.

Link to full diff

@kik-kik kik-kik marked this pull request as ready for review March 11, 2025 13:41
@kik-kik kik-kik requested review from sean-rose and scholtzan March 11, 2025 13:41
BenWu
BenWu approved these changes Mar 11, 2025
View reviewed changes
Copy link
Contributor

@BenWu BenWu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a jira ticket for this issue https://mozilla-hub.atlassian.net/browse/DENG-7977 which would address the general case but moving isp is good enough

@sean-rose
Copy link
Contributor

I'd lean toward #7119 over this.

@kik-kik
Copy link
Contributor Author

kik-kik commented Mar 11, 2025

@sean-rose , I'd be okay with that too. Except long term this means we have more points where we need to remember to make updates as columns get added to the daily table. My question here would be, what would be the risk of doing it this way? At least until we can come up with and implement a better / long term solution for this? In general, I'd argue that this specific table should only contain the client_id, sample_id, and the bit fields and nothing else. But since this is currently blocking my changes related to ToS / ToU which is high priority so my preference would be to just take the action of least friction now so that I can get that work completed.

@sean-rose
Copy link
Contributor

That makes sense, and I'm fine with you merging this to unblock downstream work, I'd just personally like to see something like #7119 be the ultimate solution. Only selectively reordering columns when a schema mismatch with incompatible column types causes an outright query error still leaves the possibility of a potentially worse outcome of this query silently doing the wrong thing if there's a mismatched column order with compatible column types and nobody noticing.

@kik-kik kik-kik added this pull request to the merge queue Mar 11, 2025
Merged via the queue into main with commit 6a7c3bb Mar 11, 2025
21 checks passed
@kik-kik kik-kik deleted the fix/out-of-order-column-baseline-clients-last-seen branch March 11, 2025 17:18
@sean-rose
Copy link
Contributor

I belatedly realized that sql_generators/glean_usage/templates/baseline_clients_last_seen_v1.schema.yaml should probably have also been updated here to reflect the new column order.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@scholtzan scholtzan scholtzan approved these changes

@BenWu BenWu BenWu approved these changes

@sean-rose sean-rose Awaiting requested review from sean-rose

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@kik-kik @dataops-ci-bot @sean-rose @scholtzan @BenWu