Open Office Hours Season 1: Session 11 - Triggers and Near-Real-Time Patterns
Triggers and Near-Real-Time Patterns in Rapidi
Your ERP has just updated a price list. Your sales team is quoting from Salesforce. How fast does that change need to reach them - and what happens if the sync fires at the wrong moment?
This article covers when to use trigger-based execution in Rapidi, when to avoid it, and how to design fast-but-safe sync patterns that keep your data consistent without overwhelming your systems.
- Trigger basics: What trigger runs are in Rapidi and how they differ from scheduled runs.
- When to trigger: The scenarios where near-real-time sync genuinely adds business value.
- When NOT to trigger: Patterns that look like they need instant sync but work better on a schedule.
- Safe patterns: How to get speed without risking data conflicts, partial writes, or cascading errors.
- Dependencies at speed: Making sure related data lands in the right order, even in near-real-time.
Watch the Full Session
This article is based on Session 11 of Rapidi's Open Office Hours training program. The full session includes a live walkthrough of trigger configuration in MyRapidi, practical examples of near-real-time patterns, and common pitfalls to avoid.
What Are Trigger Runs in Rapidi?
A trigger run is a transfer execution that starts automatically in response to a specific event - typically a data change in the source system. Unlike scheduled runs, which execute at fixed intervals regardless of whether anything has changed, trigger runs fire only when there is something to process.
This makes triggers ideal for time-sensitive data where delays have a direct business impact. But it also means triggers need more careful design, because they can fire frequently, unpredictably, and sometimes in rapid succession.
Triggers are not a replacement for schedules. They are a complement. The best integration designs use both - triggers for time-critical data, schedules for everything else.
When to Activate Trigger Runs
Trigger runs make sense when three conditions are met: the data is time-sensitive, the volume per event is small, and the business process downstream cannot wait for the next scheduled run.
Common scenarios where triggers add genuine value:
- Price changes: When your ERP updates a price list, sales teams quoting from a CRM need the new prices immediately - not in 30 minutes.
- New orders: An order placed in your CRM or webshop should appear in the ERP as soon as possible to start fulfillment.
- Inventory updates: Stock level changes that affect availability displays or order acceptance logic need near-real-time accuracy.
- Status changes: When an order ships or an invoice is posted, downstream systems need to reflect that status quickly.
In all these cases, the cost of a delay is concrete - a wrong quote, a missed order, an oversold product, or a confused customer.
When NOT to Use Triggers
Not every data flow benefits from triggers. In many cases, triggers create more problems than they solve:
- Bulk imports or migrations: Loading 10,000 records into your ERP should not fire 10,000 individual trigger runs. Use a schedule for bulk operations.
- Reference data: Product categories, units of measure, and currency tables change rarely. Scheduling these hourly or daily is more efficient.
- Data with complex dependencies: If syncing record A requires records B, C, and D to already exist in the destination, a trigger on A alone will fail. Sequential scheduled groups handle dependencies better.
- High-volume, low-urgency updates: Contact address changes, notes, and activity logs can wait for the next scheduled run without any business impact.
Rule of thumb: If nobody would notice a 15-minute delay, a schedule is the right choice. Triggers should be reserved for data where minutes matter.
Fast-but-Safe Sync Patterns
Speed and safety are not mutually exclusive, but they require deliberate design. Here are patterns that deliver near-real-time performance without compromising data integrity:
-
Debounce pattern: Instead of triggering on every individual change, collect changes over a short window (e.g., 30 seconds) and process them as a batch. This prevents rapid-fire triggers from overwhelming the destination system.
-
Filter-first pattern: Apply filters to your trigger so it only fires for records that meet specific criteria. For example, trigger only on orders with status "Confirmed" - not on every order edit.
-
Idempotent transfers: Design your transfers so that processing the same record twice produces the same result. This makes triggers safe even if they fire more often than expected.
-
Fallback schedule: Run a scheduled catch-up at a longer interval (e.g., every hour) that picks up anything the triggers may have missed. This provides a safety net without duplicating work, because Rapidi's timestamp tracking ensures records are not processed twice.
Dependency-Aware Triggering
The trickiest part of trigger design is handling dependencies. When a triggered transfer creates a record that another transfer depends on, timing becomes critical.
Consider this scenario: a new customer is created in the CRM, and immediately after, an order is placed for that customer. If both the customer sync and the order sync are triggered independently, the order may arrive at the ERP before the customer record exists - causing an error.
Strategies for managing this:
- Chained triggers: Configure the order trigger to check that the customer record exists in the destination before processing. If it does not, the order waits for the next run.
- Priority ordering: Ensure master data triggers (customers, products) always process before transactional data triggers (orders, invoices).
- Continue on Error with retry: Let the order transfer continue on error for missing dependencies, then reprocess the failed records on the next scheduled run when the customer record will be available.
Monitoring Trigger Performance
Triggers require more active monitoring than schedules because their execution is event-driven and less predictable:
- Trigger frequency: How often are triggers firing? A sudden spike may indicate a bulk operation that should have been handled differently.
- Execution time: How long does each triggered run take? If execution time grows, the trigger may be processing too many records per event.
- Error rates: Are triggered runs failing more often than scheduled runs? This often points to dependency issues or race conditions.
- Queue depth: If triggers are firing faster than they can be processed, a backlog will form. Monitor for this and consider throttling or switching to a schedule.
Common Trigger Mistakes to Avoid
- Triggering everything: Not all data needs near-real-time sync. Overusing triggers wastes API calls and creates unnecessary system load.
- Ignoring bulk scenarios: A trigger designed for single-record changes will choke on a 5,000-record import. Always have a bulk fallback.
- No dependency handling: Triggering child records without ensuring parent records exist first leads to consistent failures.
- Missing the fallback schedule: Triggers can miss events due to system downtime or API throttling. A periodic catch-up schedule ensures nothing falls through the cracks.
Watch all sessions and register for upcoming ones: rapidionline.com/resources/open-office-hours
See the full Season 1 schedule: rapidionline.com/product-updates/open-office-hours-season-1
Frequently Asked Questions
What is the difference between a trigger run and a scheduled run?
A scheduled run executes at fixed intervals regardless of whether data has changed. A trigger run fires in response to a specific event, such as a record being created or updated in the source system. Triggers provide faster sync for time-sensitive data, while schedules are better for predictable, bulk, or low-urgency data flows.
Can triggers handle large volumes of data?
Triggers are designed for small, frequent updates - not bulk operations. If a bulk import or migration triggers thousands of individual runs, it can overwhelm the system. For large volumes, use a scheduled run instead, or implement a debounce pattern that batches changes before processing.
How do I prevent triggers from firing during a bulk import?
You can temporarily disable the trigger before starting the bulk import and re-enable it afterward. Alternatively, use filters on the trigger so it only fires for records that meet specific criteria, excluding bulk-imported records. A scheduled catch-up run will process the imported records at the next interval.
What happens if a trigger fires but the dependent data is not yet available?
If a triggered transfer references a record that does not exist in the destination, it will produce an error for that record. Using Continue on Error, the transfer can complete the remaining records and the failed ones will be retried on the next run - by which time the dependent data should be available.
Should I use triggers or schedules for my integration?
Most integrations benefit from a combination of both. Use triggers for data where delays have a direct business impact - orders, price changes, inventory levels. Use schedules for everything else - contacts, reference data, historical records. Adding a periodic catch-up schedule alongside your triggers provides a safety net for any events that triggers may miss.