What Real-Time Intelligence actually is#
This is the practitioner-depth treatment of Real-Time Intelligence — the Real-Time hub as catalog, the current Eventstream connector lineup, Activator's Copy-job automation, Outbound Access Protection, and internals. For a compact orientation to the hot-path/cold-path shape and when to reach for streaming over batch, see Real-Time Intelligence in Microsoft Fabric.
Real-Time Intelligence (RTI) is Microsoft Fabric's answer to a problem that batch pipelines were never designed to solve: data that has to be acted on while it's still arriving, not after it's landed and been transformed overnight. Fabric describes RTI as a complete end-to-end solution — ingestion, transformation, storage, modeling, analytics, visualization, and automated action, all purpose-built for event-driven and streaming scenarios [S1].
That "end-to-end" claim matters because it's easy to assume real-time features are a bolt-on to a batch-first platform. They aren't. RTI ships as a first-class capability alongside Lakehouse, Warehouse, and Data Factory, with its own storage engine (Eventhouse), its own no-code ingestion layer (Eventstreams), and its own automation engine (Activator) — all discoverable through a single tenant-wide catalog called the Real-Time hub.
The platform keeps moving on two fronts at once: it's growing the set of things it can connect to (new source connectors, security options, and private networking for Eventstreams), and it's growing the set of things it can do without a human in the loop (Activator now drives Copy job runs directly, and Fabric's own AI Skill can stand up an entire Eventstream from a prompt). At the same time, the platform is adding the outbound network governance that lets security teams say yes to event-driven architecture without saying yes to unrestricted egress. This article covers all three: the moving pieces, how they fit together, where the connectors and automation have grown, and where the internals are still thin in the knowledge base.
Core concepts#
The Real-Time hub is the catalog, not the engine#
The easiest way to get RTI's shape wrong is to think of the Real-Time hub as a processing engine. It isn't — it's a discovery and cataloging layer. Fabric positions the Real-Time hub as the tenant-wide counterpart to OneLake, but for data in motion rather than data at rest: it catalogs eventstreams, KQL database tables, Microsoft-managed source streams, and Fabric or Azure platform events so people across the tenant can find, ingest, and react to streaming data in one place [S2].
Concretely, that catalog surfaces active data streams, Microsoft-managed sources — including change-data-capture (CDC) feeds from Azure SQL Database, Cosmos DB, and PostgreSQL — and Fabric workspace item events, all visible to every workload in the tenant [S1]. You don't build a stream and then separately register it somewhere; landing it in RTI makes it discoverable by default.
Because Real-Time hub is a catalog, "connecting to real-time data" in Fabric usually starts there, not in a specific workload. You browse the hub, pick a stream or table, and route it — the actual compute happens downstream in Eventstreams, Eventhouse, or Activator.
Eventstreams: the no-code ingestion and shaping layer#
Eventstreams are Fabric's no-code pipeline for collecting, transforming, and routing high-volume real-time data to multiple destinations at once. Supported sources include Apache Kafka, AWS Kinesis, Google Cloud Pub/Sub, MQTT v3.1/v3.1.1, and a purpose-built MQTT-based Real-Time Weather connector [S1]. A Fabric blog on the AI-driven weather monitoring pattern states Eventstream supports 25 source types in total [S4] — Inference: the exact figure is a snapshot at time of writing and will likely grow, since new connectors keep shipping (below).
Eventstreams aren't just a pass-through pipe. They expose built-in stream processing operators — filtering, cleansing, windowed aggregations, deduplication, and content-based routing — so data can be reshaped in flight before it lands anywhere. Those transformations can also be published as derived eventstreams: reusable, shareable outputs that other teams can subscribe to without re-authoring the same logic [S1]. A community tutorial describes the same shape slightly differently — sources feed a default stream, which optionally passes through chained transformation operators before fanning out to one or more destinations [S5]; hedge: that source is a tier-6 community write-up rather than official documentation, so treat it as a helpful mental model rather than a specification.
Eventstreams provide a no-code pipeline for collecting, transforming, and routing high-volume real-time data to multiple destinations simultaneously [S1]
What's new in the connector lineup#
Fabric has been shipping Eventstream connector updates in clusters — new sources, new security options for existing sources, and a private-networking path for all of them. As of the latest Eventstream connector update:
- Apache Kafka and Azure Service Bus source connectors have both reached general availability [S3].
- The GA Kafka connector adds SASL_SSL support alongside its existing SSL support, combining SASL-based authentication (PLAIN or SCRAM) with TLS encryption for production clusters [S3].
- Custom CA and mutual TLS (mTLS), previously preview, is now generally available across the MQTT, Kafka, AWS MSK, and Confluent Cloud connectors — customers can supply their own CA and client certificates from Azure Key Vault for mutually authenticated encrypted connections [S3].
- An enhanced Azure IoT Hub source connector is in preview, built on the Kafka Connect framework. It preserves full device-to-cloud message metadata (device ID, enqueued time, auth method) by copying it into user metadata, prefixing system metadata keys with
__src__so they're distinguishable from user-defined properties [S3]. That metadata can then be pulled out inside an Eventstream SQL operator with aGETMETADATAPROPERTYVALUEfunction, enabling per-device latency monitoring or conditional routing by device identity [S3]. - A new Oracle Database CDC source connector is in preview, streaming changes from on-premises or cloud-hosted Oracle databases into Eventhouse, Lakehouse, Activator, or custom endpoints [S3].
- The HTTP source connector now supports pagination in preview, natively walking multi-page REST API responses via either page-based (page number/size/offset) or cursor-based (continuation token) configurations [S3].
- Private network support (vNet / on-premises) for Eventstream streaming connectors has moved from preview to general availability — connectors can reach sources in private Azure virtual networks or on-premises environments without public internet exposure [S3]. Under the hood this runs on a Streaming virtual network data gateway, a Fabric construct that provisions a bridge Azure virtual network/subnet so the connector executes inside the customer's network, with connectivity to the actual private source established via VPN, ExpressRoute, private endpoints, or peering as appropriate [S3].
Preview vs. GA matters here because it changes what you can commit to in a production design: the Oracle CDC connector, the enhanced IoT Hub connector, and HTTP pagination are all preview — build against them, but don't treat their current behavior as a stable contract. Kafka/Service Bus GA, mTLS GA, and private-networking GA are safe to depend on for production architectures [S3].
Building an Eventstream from a prompt, not a canvas#
Microsoft has published an Eventstream AI Skill — distributed as a skills-for-fabric package on GitHub — that generates and deploys a complete Eventstream pipeline from a single natural-language prompt describing sources, filter conditions, and a destination [S4]. Manually building one Eventstream pipeline through the portal (choosing a source type, configuring properties, wiring operators, choosing a destination, publishing) is characterized in the same post as roughly 40+ discrete portal interactions taking several minutes per source [S4]; hedge: that figure describes one vendor's demo walkthrough, not a controlled usability study, so treat it as illustrative rather than a benchmark.
The pattern the post demonstrates is a real-time weather monitoring architecture: the Real-Time Weather Eventstream source is a zero-configuration source needing only a location name or lat/long — no Azure subscription, Event Hub provisioning, or connection credentials [S4]. The recommended shape is one real-time source per monitored location feeding a Filter operator with a threshold condition, with matching events routed into a single shared Eventhouse KQL table tagged by originating entity [S4]. Deploying that pattern with many parallel sources, a shared filter, and multiple destinations is described as the same authoring effort as deploying a single-source version, when done through the AI Skill rather than manual configuration [S4] — Inference: this is a claim about authoring effort via natural-language generation, not about the runtime cost of running many parallel sources, which is a separate (and here, ungrounded) question.
Under the hood, the AI Skill resolves workspace and resource identifiers, constructs the Eventstream topology definition, base64-encodes it, and deploys it via Fabric REST APIs rather than requiring manual portal configuration [S4].
Activator: turning conditions into actions#
Activator is RTI's automation layer. It watches data flowing through the Real-Time hub, KQL queries, or Power BI reports, and fires actions automatically when a threshold, pattern, or KQL-defined condition is met. Those actions include sending notifications, launching a Fabric pipeline, Spark job, or Dataflow run, calling User Data Functions, or triggering Power Automate workflows [S1]. A Fabric blog on its newest capability frames Activator plainly: it's a real-time engine that monitors data and reacts to events, in direct contrast to Data Factory's traditional fixed-schedule execution model [S6].
This is the piece that closes the loop between "we detected something" and "something happened as a result" — without a human watching a dashboard and clicking a button.
Activator can now drive Copy jobs directly — and it's GA#
The newest addition to Activator's action list is significant enough to call out on its own: Fabric Activator can now directly invoke a Copy job as a rule action, without requiring an intermediate Data Factory pipeline, and this capability is Generally Available [S6]. Before this, triggering a data-movement job from an event meant wiring Activator to a pipeline that then ran the copy — an extra layer of indirection. Now the Copy job is a first-class Activator action.
An Activator rule that invokes a Copy job can fire from conditions like a file being created in a monitored location, a table being updated (for example in OneLake), or a metric crossing a defined threshold [S6]. Once such a rule is enabled, it continuously monitors its condition and runs the Copy job automatically only when the condition is actually met — not on every scheduled interval [S6]. That's the core of event-driven orchestration: replacing "check every N minutes whether there's new data" with "run the moment there actually is new data."
This is GA, not preview — it's safe to build production event-driven ingestion pipelines around Activator-triggered Copy jobs today. Inference: the broader pattern of Activator-triggered orchestration (Copy jobs, pipelines, Spark jobs, Dataflows) is a natural foundation for a general "event-driven pipeline orchestration" design, which is a good next design candidate now that the Copy job path specifically has reached GA.
Analytics without moving the data#
RTI's query surface is deliberately multi-language. Fabric notebooks can operate directly over Eventhouse data using KQL, T-SQL, Python, or Spark, without requiring data movement — which means exploration, transformation, and machine learning can happen inside a single authoring surface instead of exporting data to a separate analytics tool first [S1].
RTI also supports geospatial analytics through Map, a built-in visualization experience that layers bubbles, heatmaps, polygons, and 3D extrusions over Eventhouse or Lakehouse data, with configurable KQL refresh intervals so the map itself updates live as new events arrive [S1].
How it works: best practices#
Route through Real-Time hub instead of hand-wiring point-to-point connections#
Rule: land a new streaming source in the Real-Time hub / Eventstreams layer rather than piping it directly into a single downstream workload.
Why: the hub is what makes a stream discoverable tenant-wide, and Eventstreams can fan the same source out to multiple destinations simultaneously [S1]. A point-to-point connection (source straight into one KQL database, say) works for the first consumer but has to be re-built for every subsequent one.
Example — wrong vs. right shape:
# Wrong: point-to-point, not reusable
Kafka topic ──▶ single Eventhouse table
(any second consumer needs its own separate Kafka connection)
# Right: fan out through an eventstream
Kafka topic ──▶ Eventstream
├──▶ Eventhouse table (durable, queryable)
├──▶ Activator (threshold alerting, or now: a Copy job run)
└──▶ derived eventstream (reusable, filtered/deduped output
that other teams can subscribe to) <sup id="cite-1"><a href="#src-1" class="cite">[S1]</a></sup> <sup id="cite-6"><a href="#src-6" class="cite">[S6]</a></sup>
Push transformation logic into the eventstream, not downstream consumers#
Rule: do filtering, deduplication, and windowed aggregation in the eventstream itself using its built-in operators, rather than shipping raw events to every consumer and repeating the same cleanup logic in each one.
Why: Eventstreams expose filtering, cleansing, windowed aggregations, deduplication, and content-based routing as built-in operators, and the shaped output can be published as a derived eventstream that's reusable across consumers [S1]. Doing the same cleanup redundantly in Activator rules, notebooks, and dashboards means three places to keep in sync every time the logic changes.
# Before: raw events fanned to three consumers, each re-implementing cleanup
raw_topic ──▶ Eventhouse (dedupes on write)
raw_topic ──▶ Activator (filters bad readings inline in the rule)
raw_topic ──▶ Notebook (re-filters again in a Spark job)
# After: cleanup happens once, in the eventstream
raw_topic ──▶ Eventstream (dedupe + filter + window)
└──▶ derived_eventstream ──▶ Eventhouse
──▶ Activator
──▶ Notebook
Default to Direct Ingestion; add Event Processing only when you need it#
Rule: start a new Eventstream with Direct Ingestion (no transformation operators), and only introduce Event Processing transforms when you specifically need volume reduction, enrichment, or content-based routing.
Why: a community Eventstream tutorial characterizes Direct Ingestion as carrying lower latency and lower capacity-unit cost than routing events through transformation first, since every operator in the chain costs compute [S5]. The same source recommends pushing filtering or aggregation to query time by default, and only adding Event Processing operators when reducing volume, enriching records, or routing by content is a real requirement [S5]. Hedge: this guidance comes from a tier-6 community source, not Microsoft documentation — treat it as a reasonable capacity-planning heuristic to validate against your own workload's cost and latency numbers, not a documented Microsoft best practice.
# Start here (Direct Ingestion) — cheapest, lowest latency
Source ──▶ Eventstream (no operators) ──▶ Eventhouse
(filter/aggregate at query time)
# Only add operators when you have a concrete reason
Source ──▶ Eventstream ──▶ [Filter: drop malformed] ──▶ [Aggregate: 5-min window]
└──▶ Eventhouse
The same source notes that Eventstream's transformation operators (filter, aggregate, group-by, join, manage fields, convert) can be chained, and that filtering as close to the source as possible avoids paying capacity units to transform data that will be discarded anyway [S5]. Hedge: again tier-6 community guidance — a sound engineering principle, but not a Microsoft-documented cost model.
Let event-driven triggers replace polling schedules where they're available#
Rule: where a workflow can start from a Fabric or Azure Blob Storage event surfaced in the Real-Time hub, wire it to an Activator rule instead of a fixed-interval scheduled trigger.
Why: Fabric events and Azure Blob Storage events exposed in the Real-Time hub can trigger Activator rules to start Spark jobs or Dataflows, enabling fully event-driven orchestration that needs no polling schedule [S1]. That principle now extends to data movement specifically: Activator-triggered Copy jobs run only when their condition is actually met, not on a fixed schedule [S6]. Data Factory's own storage-event triggers are built on the same foundation: the event flow runs through an eventstream and the trigger rule itself lives in Data Activator, so trigger management for those pipelines happens partly outside the pipeline editor [S7].
If you're debugging a Data Factory storage-event trigger and can't find the rule logic in the pipeline canvas, that's expected — the condition lives in Activator, not the pipeline definition [S7].
Govern cross-workspace event flows with Outbound Access Protection (preview)#
Rule: if a workspace consumes Fabric or Azure events (or Activator actions, or Eventstream destinations) that live in a different workspace, expect that traffic to be blocked by default once Workspace Outbound Access Protection (OAP) is enabled — and plan the data connection rule that re-allows it as part of the design, not as a break-fix afterward.
Why: OAP for events brings the same default-deny outbound network model already used elsewhere in Fabric to event consumption specifically. When OAP is enabled on a workspace, all outbound connections from items in that workspace are blocked by default unless explicitly permitted through the workspace's data connection rules [S8]. Event consumption within the same workspace is always allowed regardless of the OAP setting — OAP only affects traffic that crosses a workspace boundary [S8]. Concretely, an Activator alert or Eventstream in an OAP-protected workspace can no longer consume Job or OneLake events emitted from a different workspace unless a data connection rule for the Real-Time Events connector explicitly allows it [S8].
OAP for Fabric/Azure events, Eventstream, Activator, and Real-Time Dashboard is currently preview, and is explicitly framed by Microsoft as the first stage of OAP integration for the Fabric eventing ecosystem — the same post notes plans to extend coverage to Power BI reports and maps next, which has not shipped yet [S8] [S9]. Design for it, but track it as evolving surface area, not a finished control.
# Before OAP: Workspace B's Activator freely consumes Workspace A's OneLake events
Workspace A (OneLake events) ────────▶ Workspace B (Activator rule)
[always worked, no config needed]
# After enabling OAP on Workspace B: blocked by default
Workspace A (OneLake events) ───X───▶ Workspace B (Activator rule, OAP on)
[blocked — cross-workspace, no rule yet]
# Fix: add a data connection rule for the Real-Time Events connector
Workspace A (OneLake events) ────────▶ Workspace B (Activator rule, OAP on)
[allowed — explicit data connection rule for
Real-Time Events connector added] <sup id="cite-8"><a href="#src-8" class="cite">[S8]</a></sup>
Two operational details matter once you've turned OAP on for a workspace that already has cross-workspace consumers configured. First, Real-Time hub doesn't delete those existing consumers when OAP is switched on — it detects the policy change and moves the affected configurations into a Paused state, visible on the Fabric events or Azure events page with a reason shown via View details [S8]. Second, paused configurations keep retaining their events for up to seven days after the pause begins; if the access rule isn't corrected within that window, the retained events are dropped rather than delivered [S8]. That seven-day window is your real remediation deadline, not an indefinite grace period.
Resuming a paused cross-workspace event consumer isn't automatic even after you add the missing data connection rule — the consumer configuration has to be deleted and recreated [S8]. Budget for that manual step in any OAP rollout runbook.
OAP's reach across RTI's individual items breaks down like this:
| Item | Behavior under Workspace OAP (preview) |
|---|---|
| Eventstream | Continues routing to RTI/Fabric items in the same workspace; cross-workspace destinations and external service/database/API destinations blocked by default [S9] |
| Activator | Actions targeting Fabric items (notebooks, Spark, pipelines, User Data Functions, Dataflows) in the same workspace always allowed; other workspaces blocked unless explicitly permitted [S9] |
| Activator → Teams | Notifications restricted to same tenant, configurable per workspace via a Teams connection kind [S9] |
| Activator → email | Blocked by default, limited to same-tenant recipients [S9] |
| Activator → Power Automate | Blocked outright; cannot currently be re-enabled via data connection rules [S9] |
| Real-Time Dashboard | Most day-to-day viewing/monitoring is unaffected — RTD's outbound access model is described as intentionally simple [S9]; Copilot-assisted tile add/edit, Copilot-assisted exploration, and sharing via the editor's own identity are blocked [S9] |
Finally, OAP for events is one of three layered network controls Microsoft positions together, and it's worth knowing which one governs which direction of traffic: OAP governs whether a consumer workspace can reach event sources elsewhere; workspace-level private links govern whether other workspaces can reach a source workspace that blocks public access; and tenant-level private links govern whether external Azure events (e.g. Azure Blob Storage events) can reach a tenant that blocks public internet access [S8]. A related architectural detail: OAP for events is enforced only on the consumer's workspace — the OAP configuration of the workspace producing the events has no bearing on whether the flow is allowed [S8].
Internals#
Architecture & design#
RTI's architecture separates three concerns that are easy to conflate: cataloging (Real-Time hub), ingestion/shaping (Eventstreams), and reaction (Activator) — sitting on top of a durable storage and query layer (Eventhouse, covered in Fabric's dedicated Eventhouse/KQL capability). The Real-Time hub is explicitly the discovery layer, cataloging eventstreams, KQL database tables, Microsoft source streams, and platform events across the tenant [S2], while Eventstreams own the actual data-shaping work with their filter/dedupe/window/route operators [S1].
Activator is architected as a condition-and-action engine that can subscribe to three different kinds of input — Real-Time hub streams, KQL queries, or Power BI report data — and dispatch to five different kinds of action: notifications, pipeline runs, Spark job runs, Dataflow runs, User Data Function calls, or Power Automate flows [S1]. Copy job invocation is now a sixth action kind, and it's architected the same way as the others: continuous condition monitoring with the action firing only on a met condition, not on a schedule [S6]. That breadth is what lets a single Activator rule sit downstream of either a live stream or a scheduled analytical query and still trigger the same class of response.
The connector layer has its own architectural wrinkle worth calling out: private-network connectivity for Eventstream sources is not a simple firewall allow-list. It runs through a Streaming virtual network data gateway, a Fabric-managed construct that provisions a bridge Azure virtual network/subnet so the connector process actually executes inside the customer's network boundary, with the last-mile connection to the private source itself handled by whatever the customer already has — VPN, ExpressRoute, private endpoints, or network peering [S3]. That's a materially different design than a managed egress IP allow-list, and it's why the capability needed its own GA milestone separate from the connectors it serves.
Outbound Access Protection adds a fourth architectural layer on top of the three above: a workspace-scoped, default-deny network policy that governs whether cross-workspace event traffic is allowed at all, independent of what the source or destination items themselves support [S8]. The enforcement point is specifically the consumer workspace's policy, not the producer's [S8] — which means the same event source can be freely consumable by one workspace and blocked for another, purely based on each consumer's own OAP configuration and data connection rules.
RTI also integrates with Fabric data agents, which layer conversational analytics and workflow orchestration on top of Eventhouse data. That complements Activator's threshold-based automation with reasoning over both live and historical datasets — Activator answers "did condition X happen," data agents can answer open-ended questions across the same data [S1].
How it works internally#
Part of this layer is grounded, part is still thin. On delivery semantics: Eventstream provides at-least-once delivery, meaning duplicate events are possible — for example during connector restarts. The recommended mitigation is deduplicating at the destination: KQL's arg_max() keyed by event ID for Eventhouse, or Delta MERGE logic keyed on event ID for Lakehouse [S5]. Hedge: this delivery-semantics claim comes from a tier-6 community tutorial rather than first-party Microsoft documentation, so treat the at-least-once characterization as a practitioner's working understanding to validate for your own connector type, not a documented SLA.
On windowing: the same community source describes four temporal windowing types with distinct semantics — tumbling (non-overlapping fixed intervals), hopping (overlapping fixed intervals, useful for smoothed/moving-average metrics), session (dynamic duration based on gaps between events), and sliding (a continuously advancing fixed-duration window for real-time threshold detection) [S5]. Hedge: same tier-6 caveat applies — useful as a working vocabulary, not a specification.
Beyond delivery semantics and windowing, the deeper execution path is still a gap: how Eventstreams schedule and checkpoint their operator pipeline internally, and how Activator evaluates threshold/pattern/KQL conditions against a live stream under the hood, are not covered by any verified L4/L5 source yet.
Coming soon — the operator-pipeline scheduling and checkpointing internals, and Activator's live condition-evaluation engine, aren't in the knowledge base yet at engineering-internals depth. It needs an L4/L5 source such as an Azure/Fabric engineering blog or the Kusto (Azure Data Explorer) whitepaper documenting how Eventstreams' operator pipeline is actually executed and checkpointed, and how Activator evaluates conditions against a live stream. Tracked in content/queue.md.
Performance characteristics#
A community tutorial offers cost-shape guidance rather than measured benchmarks: Direct Ingestion (no transformation operators) is described as carrying lower latency and lower capacity-unit cost than routing events through Event Processing transforms first, because each additional operator in the chain consumes compute [S5]. The same source recommends monitoring end-to-end latency, event throughput drop relative to the source rate, error rate from schema mismatches, and pending-event queue depth as health signals — a steadily growing queue is called out as the signal that a destination is falling behind and the stream needs scaling or simplified transforms [S5]. Hedge: both of these are tier-6 community characterizations, not Microsoft-published benchmark numbers — there is still no verified measured throughput figure (events/sec by source type, or Activator rule-evaluation latency at scale) in the knowledge base.
Coming soon — measured throughput and latency benchmarks aren't in the knowledge base yet. It needs an L4/L5 source such as a Microsoft engineering blog or benchmark publishing measured Eventstream throughput numbers (events/sec by source type, end-to-end latency from ingestion to Activator firing) or Activator rule-evaluation latency at scale. Tracked in content/queue.md.
Worked example: alerting on a live IoT stream and keeping the history queryable#
Consider a fleet-monitoring scenario: vehicles publish telemetry (location, speed, fuel level) over MQTT, and you need both (a) an immediate alert when a vehicle goes offline or reports a critical fault, and (b) a durable, queryable history for trend analysis and a live map.
- Ingest. Point an Eventstream at the MQTT source — Eventstreams natively support MQTT v3.1/v3.1.1 [S1].
- Shape once. Inside the eventstream, apply a filter to drop malformed readings and a deduplication step for retried messages, then publish the cleaned result as a derived eventstream so both downstream consumers read the same shaped data [S1].
- Fan out.
- Route the derived eventstream to an Eventhouse table for durable KQL storage — this is what powers historical trend queries and the live Map visualization layer with its configurable refresh interval [S1].
- Route the same derived eventstream to Activator, with a rule that fires when a vehicle's "last seen" gap exceeds a threshold — Activator can then trigger a User Data Function to page an operator, kick off a Power Automate flow to open a ticket, or now, directly run a Copy job to land the flagged vehicle's recent telemetry window into a separate investigation lakehouse [S1] [S6].
- Investigate. When an incident happens, open a Fabric notebook against the Eventhouse table using KQL, T-SQL, Python, or Spark — no data movement required — to correlate the flagged vehicle's recent readings with fleet-wide patterns [S1].
- If the investigation workspace is OAP-protected, the Activator-triggered Copy job landing data cross-workspace needs an explicit data connection rule for the Real-Time Events connector, or the flow is blocked by default and the consumer configuration is paused [S8].
Inference: the specific choice of "gap exceeds threshold" as the Activator condition, the OneLake-fallback investigation lakehouse, and the four/five-step ordering above are a synthesized example, not a documented reference architecture; each individual capability it exercises (MQTT ingestion, dedup/filter operators, derived eventstreams, Eventhouse storage, Activator actions including Copy job invocation, OAP's cross-workspace gating, and multi-language notebook queries) is grounded in the cited claims.
// Example KQL against the Eventhouse table from a notebook — illustrative syntax,
// not a claim about a specific dataset's schema. Inference: query shape only.
VehicleTelemetry
| where ingestion_time() > ago(1h)
| where FuelLevel < 10
| summarize LowFuelEvents = count() by VehicleId
| order by LowFuelEvents desc
Inference: deduplicating this table on write would follow the community-sourced arg_max()-by-event-ID pattern [S5] — for example VehicleTelemetry | summarize arg_max(ingestion_time(), *) by EventId — shown here as an illustrative pattern, not a verified product-documented syntax.
What goes wrong#
The main antipattern surfaced in the knowledge base isn't a coding mistake — it's an organizational one: treating Data Factory's event-driven triggers as if they're fully self-contained inside the pipeline editor. Because the event flow for a storage-event trigger runs through an eventstream and the trigger rule itself lives in Data Activator, teams that only look at the pipeline canvas when debugging a trigger will miss where the actual condition logic lives [S7]. If a storage-event-triggered pipeline isn't firing as expected, check the Activator rule and the eventstream routing before assuming the pipeline definition is broken.
A second, newer antipattern is rolling out Workspace OAP without a remediation plan for existing cross-workspace consumers. Because OAP pauses rather than deletes affected configurations, and paused events are only retained for seven days before being dropped [S8], enabling OAP on a workspace with live cross-workspace Activator or Eventstream consumers — without immediately auditing and re-permitting them via data connection rules — risks silent data loss once that window elapses, not just a loud failure you'd catch immediately.