7.1 Data Integration Pipeline Design

Why Data Integration Design Shows Up on AZ-305

Every enterprise Azure environment eventually has to move data from where it is created to where it is needed — from an on-premises enterprise resource planning (ERP) system into a cloud data warehouse, from a line-of-business SQL Server into a reporting layer, or from dozens of SaaS APIs into one analytical store. The official AZ-305 skill "Recommend a solution for data integration" sits inside the Data Storage domain (20-25% of the exam) and is one of the most scenario-heavy topics on the test, because it forces you to reconcile three things at once: where the source data lives (cloud, on-premises, or SaaS), how it must be transformed, and how the pipeline should be triggered and monitored. Expect case-study questions that describe a hybrid data estate and ask you to pick the orchestration engine, the integration runtime, and the trigger type — not just name a service.

Core Service: Azure Data Factory

Azure Data Factory (ADF) is Azure's cloud-native extract, transform, load / extract, load, transform (ETL/ELT) orchestration service. It does not store or process data itself in most cases — it coordinates movement and transformation across other compute and storage services. Four building blocks matter for the exam:

• Pipelines — a logical grouping of activities that together perform a task (for example, copy → transform → load).
• Activities — the individual steps inside a pipeline: Copy Activity (moves data between 100+ connectors), Data Flow (a visual, Spark-backed, no-code transformation activity called Mapping Data Flow), Lookup, Stored Procedure, Execute Pipeline, and Web/Webhook activities.
• Linked Services and Datasets — a Linked Service defines the connection to a data store (like a connection string with authentication); a Dataset points to a specific structure within it (a table, folder, or file).
• Triggers — Schedule triggers (fixed calendar cadence), Tumbling Window triggers (fixed-size, non-overlapping time slices with built-in retry and dependency chaining — the right answer whenever a scenario mentions backfill or "process each hour's data exactly once"), and Event-based triggers (fire on blob creation/deletion in a storage account).

Integration Runtime (IR) is the compute infrastructure that actually executes the activities, and picking the correct IR type is one of the most tested design decisions:

Azure Synapse Pipelines: Same Engine, Different Home

Azure Synapse Pipelines uses the identical underlying ADF orchestration engine, but it is embedded directly inside an Azure Synapse Analytics workspace alongside SQL pools and Apache Spark pools. The exam-relevant distinction is architectural placement, not capability:

• If the organization is standardizing on a single Synapse Analytics workspace as its unified analytics platform (SQL + Spark + pipelines in one place, one workspace-level security boundary, one Git integration), recommend Synapse Pipelines so data engineers do not have to manage two separate orchestration surfaces.
• If data integration needs to fan out to many non-analytics destinations (Azure Functions, Logic Apps, on-premises systems, SaaS APIs) independent of any analytics workspace, or the organization does not otherwise need Synapse, recommend standalone Azure Data Factory.

A forward-looking fact worth knowing for a 2026 sitting: Microsoft has been steering new analytics builds toward Microsoft Fabric, whose Fabric Data Factory is described as the next generation of both ADF and Synapse Pipelines, and the classic Azure Synapse Spark runtime (3.4) reaches end of support on March 31, 2026 with Microsoft recommending migration to Fabric. The AZ-305 objective itself stays generic ("recommend a solution for data integration"), and ADF/Synapse Pipelines remain fully supported GA services, so exam scenarios can still validly point to either — but if a scenario explicitly says the customer wants a single SaaS platform combining ingestion, warehousing, and Power BI with no infrastructure to manage, Fabric is the intended design answer over classic Synapse.

Exam Scenario Walkthrough

A hospital network runs its billing system on an on-premises SQL Server behind a corporate firewall with no public endpoint. Requirements: (1) copy billing records into Azure SQL Database nightly, (2) apply a no-code transformation that masks patient identifiers before landing, (3) guarantee each night's batch is processed exactly once even if a run fails and is retried, and (4) the destination environment already runs Synapse Analytics for downstream Spark analytics. The design should include a Self-Hosted Integration Runtime installed inside the hospital's network (because the source has no public endpoint), a Mapping Data Flow for the masking transformation, a Tumbling Window trigger (for guaranteed-once, retryable nightly processing), and orchestration through Synapse Pipelines rather than standalone ADF, since the destination workspace already centralizes analytics.

Takeaways

• Azure Data Factory and Synapse Pipelines share one orchestration engine; the choice is about workspace consolidation, not feature gaps.
• Any source or sink without a public endpoint requires a Self-Hosted Integration Runtime; Azure IR alone cannot reach it.
• Tumbling Window triggers are the answer whenever a scenario needs backfill, dependency chaining, or "exactly once per interval" semantics.
• Azure-SSIS IR is the tested answer for migrating legacy SSIS packages instead of rewriting pipelines from scratch.
• Know that Fabric Data Factory is Microsoft's stated direction beyond classic Synapse pipelines, useful for scenarios that explicitly ask for a unified SaaS analytics platform.