8.3 Notebook & SQL Editor Visualizations

8.3 Notebook & SQL Editor Visualizations

Exam Focus: Databricks provides versatile visualization capabilities across both interactive Notebooks and the dedicated Databricks SQL (DBSQL) Editor. Candidates must understand the 1,000-row rendering preview limit in notebook display(), parameter binding syntax (:parameter_name), migration paths from legacy SQL dashboards to modern AI/BI Dashboards, workspace permission levels, and background schedule/email distribution configuration.

Ad-Hoc Visualization in Databricks Notebooks

Data analysts frequently perform exploratory data analysis (EDA) within Databricks Notebooks using PySpark, SQL, Python, or R. Calling the native display() function on a PySpark DataFrame—or executing a %sql query cell—renders an interactive tabular results grid directly beneath the cell.

# PySpark notebook ad-hoc exploration with display()
df_sales = spark.table("main.gold_analytics.monthly_sales")
display(df_sales)

To construct an inline visualization within a notebook:

1. Click the + icon at the top of the display() results table.
2. Select Visualization to open the embedded chart config editor.
3. Select the desired chart type (bar, line, scatter, box plot, histogram, or pivot table) and assign columns to X-axis, Y-axis, and grouping dimensions.

The 1,000-Row Notebook Preview Limit

CRITICAL EXAM CONCEPT: By default, the interactive notebook display() table and inline visualization preview enforce a client-side rendering limit of 1,000 rows.

When a query scanning a 50-million row Delta table is executed via display(), the SQL Warehouse cluster computes aggregations and transformations across all 50 million rows. However, only the first 1,000 rows of the final result set are transferred to the web browser DOM preview. Analysts must ensure that summary metrics are calculated server-side in SQL/PySpark (GROUP BY, SUM, COUNT) rather than relying on browser-side visual grouping of raw unaggregated rows.

---

Databricks SQL Editor & Parameterized Queries

The Databricks SQL (DBSQL) Editor is the dedicated interface for authoring, testing, and optimizing SQL queries. After running a query in SQL Editor, analysts can add multiple named visualization tabs to a single query result set without re-executing the underlying SQL code.

Authoring Dynamic Query Parameters

SQL Editor supports dynamic query parameters, allowing users to alter filter criteria directly from UI input widgets. Parameters are declared in SQL code using the :parameter_name syntax (or legacy {{parameter_name}} syntax).

-- Parameterized SQL query in Databricks SQL Editor
SELECT 
    order_date,
    product_category,
    SUM(order_amount) AS daily_revenue,
    COUNT(DISTINCT customer_id) AS active_customers
FROM main.gold_analytics.fact_orders
WHERE order_date BETWEEN :start_date AND :end_date
  AND region = :region_param
GROUP BY order_date, product_category
ORDER BY order_date ASC;

When :start_date, :end_date, or :region_param are included in the query text, Databricks SQL Editor automatically generates corresponding widget controls beneath the SQL editor pane:

• Text / Number Parameters: Free-form text input or numeric boundary inputs.
• Dropdown Parameters: Single-select or multi-select dropdown menus populated via hardcoded lists or populated dynamically from a secondary SQL query.
• Date & Date Range Parameters: Interactive calendar pickers emitting formatted date strings.

---

Workspace Permissions & Legacy Asset Migration

Workspace Access Control

Access to queries, notebooks, and SQL Editor visualizations is governed by workspace access control levels:

Migrating Legacy DBSQL Dashboards to AI/BI Dashboards

Legacy Databricks SQL Dashboards bound each visual widget to a standalone, isolated query object in the workspace. Modern AI/BI Dashboards replace this fragmented model with centralized dashboard datasets. Databricks provides automated migration tools to convert legacy DBSQL dashboard queries into AI/BI Dashboard datasets, consolidating governance under Unity Catalog.

---

Scheduled Refreshes, PDF Subscriptions & Alerts

To deliver timely insights without manual execution, Databricks supports automated scheduling and alerting:

Scheduled Execution & Email Subscriptions

Published AI/BI Dashboards and SQL Editor queries can be configured with background refresh schedules.

• Schedule Frequency: Configurable from interval cadences (as frequent as every 1 minute) to cron-based schedules (e.g., Every Monday at 08:00 UTC).
• Email Subscriptions: Authors add user or group email addresses to the subscription list. Upon schedule completion, Databricks automatically generates a high-resolution PDF snapshot or PNG export of the published dashboard and emails it directly to subscribers.
• Compute Efficiency: Background refreshes execute on designated Databricks SQL Warehouses (Serverless or Pro), utilizing query result caching when underlying data has not changed.

SQL Editor Alerts

In addition to scheduled dashboards, SQL Editor supports Alerts. An alert monitors a query result at scheduled intervals and triggers notifications (via Email, Slack, Microsoft Teams, or PagerDuty webhooks) when a metric crosses a defined threshold (e.g., failed_transactions > 50).