3.3 Consecutive System Exceptions & Graceful Process Termination
Key Takeaways
MaxConsecutiveSystemExceptions on the Constants sheet stops a job after that many System Exceptions in a row; the template value 0 disables it.
The counter rises on every System Exception and resets to 0 on success and on a business exception, because a business exception proves the applications work.
The check runs at the start of Initialization: it throws, Initialization's catch sets SystemException, and the state machine goes to End Process.
With ShouldMarkJobAsFaulted = False (template value) the stopped job still shows Successful; set it to True to make the stop visible as Faulted.
Stopping early leaves unattempted queue items in New status, so recovery after an outage needs no re-upload.
3.3 Consecutive System Exceptions & Graceful Process Termination
Core Concept: Retries handle a single glitch. They do not protect you when a whole application is down. REFramework adds a circuit breaker, the
MaxConsecutiveSystemExceptionsconstant, that stops the job after a run of System Exceptions in a row. The remaining queue items then stay New until the outage is fixed.
Picture a performer that starts at midnight with 5,000 queue items. At 01:00 the ERP it logs into goes offline. Without a limit, the robot takes item 1, fails, closes and reopens the applications, fails again when the retry copy arrives, and keeps doing this for every item. By morning the whole queue is Failed, every retry has been used up, and someone has to re-upload thousands of items.
The Two Pieces: A Constant and a Counter
| Element | Where it lives | Behavior |
|---|---|---|
MaxConsecutiveSystemExceptions | Constants sheet of Config.xlsx | Template value 0, which disables the check. Set it to a small number, such as 3 to 5, for production performers. |
ConsecutiveSystemExceptions | Variable in Main.xaml, passed to SetTransactionStatus.xaml as an In/Out argument | Incremented by 1 on every System Exception; reset to 0 on a successful transaction and on a business exception. |
Why a business exception resets the streak
To raise a business rule exception, the robot has to reach the application, log in, open the record, and read its data. A business exception is therefore proof that the environment works, even though that particular record is invalid. So the streak of technical failures is broken and the counter goes back to 0. Exam questions like to mix the two exception types to test this.
Where the Check Happens: Initialization, Not Process Transaction
The standard template does not add an extra transition out of Process Transaction. The flow is:
- A transaction throws a System Exception.
SetTransactionStatus.xamlsets the status, takes a screenshot, closes the applications, and increments the counter. - The System Exception transition routes back to Initialization as usual.
- At the start of Initialization, inside the Try block, the template checks:
CInt(Config("MaxConsecutiveSystemExceptions")) > 0 AndAlso
ConsecutiveSystemExceptions >= CInt(Config("MaxConsecutiveSystemExceptions"))
- If the condition is true, it throws an exception whose message comes from
ExceptionMessage_ConsecutiveErrors. - Initialization's own Catch stores that exception in
SystemException, so the System Exception transition from Initialization sends the state machine to End Process. - End Process closes (or kills) the applications and the job ends.
Successful or Faulted? The ShouldMarkJobAsFaulted Constant
Because the framework catches the exception itself, the job normally ends with the Orchestrator status Successful, even though it stopped early. The Constants sheet includes ShouldMarkJobAsFaulted (template value False). When it is True, End Process rethrows the Initialization exception, so the job ends as Faulted. That matters for operations teams, because Orchestrator alerts and many monitoring rules react to faulted jobs, not to warnings buried in logs.
Outage Example with a Limit of 3
| Dimension | No limit (0) | MaxConsecutiveSystemExceptions = 3 |
|---|---|---|
| Items attempted during a 4-hour ERP outage | Every item in the queue | 3 items, then the job stops |
| Items left New | None | All unattempted items |
| Retry copies used | All of them | Only for the 3 failed items |
| Job status | Successful, with thousands of failures in the logs | Successful, or Faulted if ShouldMarkJobAsFaulted = True |
| Recovery work | Re-add or retry thousands of items | Fix the outage; the next job continues from the queue |
A queue trigger can start a new job while New items remain, so a stopped performer may be started again later automatically. That is usually the behavior you want once the system is back. If the outage lasts longer, the new job will stop after three failures again rather than burning the queue.
Retries Versus the Circuit Breaker
| Question | Queue Auto Retry or MaxRetryNumber | MaxConsecutiveSystemExceptions |
|---|---|---|
| What does it protect against? | A single item failing because of a short glitch | A whole application or network being down |
| What does it count? | Attempts for one item | System Exceptions in a row across items |
| What happens when it trips? | The item ends as Failed | The job stops through End Process |
| Template value | Queue setting; MaxRetryNumber = 0 | 0 (disabled) |
Design Tips
- Pick the limit from how the application fails. Three in a row is common when each item touches the same system. Items that each hit a different external site may need a higher limit or none.
- Log the counter value when it trips, and consider
ShouldMarkJobAsFaulted = Trueso the stop is visible in Orchestrator. - Do not reset the counter in Process.xaml or in custom retry logic. Resetting it defeats the circuit breaker.
An unattended REFramework process has MaxConsecutiveSystemExceptions set to 3. It hits two System Exceptions in a row, and then the third transaction throws a BusinessRuleException. What is the consecutive exception counter after the third transaction?
0, because a BusinessRuleException confirms that the underlying applications are functional and responsive, resetting the consecutive counter
1, because any exception resets the counter to 1 regardless of type
3, because business exceptions count toward the consecutive system exception threshold
2, because business exceptions are ignored and the existing consecutive system exception count is frozen
Why is stopping a performer through MaxConsecutiveSystemExceptions good practice when the core ERP it uses suffers an outage?
It automatically notifies the ERP vendor to reboot the remote server cluster
It deletes all failing queue items from Orchestrator to maintain a high process success rate
It converts all technical system errors into business rule exceptions for easier reporting
It stops the robot from needlessly failing the entire queue, preserving unattempted items in New status until the outage is resolved
In the standard REFramework template, where is the MaxConsecutiveSystemExceptions limit enforced, and what happens when it is reached?
In Process Transaction, through a dedicated transition that skips Initialization and goes straight to End Process.
In Get Transaction Data, which stops fetching items but keeps the applications open.
At the start of Initialization, which throws an exception that its own Try Catch stores in SystemException, so the Initialization System Exception transition leads to End Process.
In Orchestrator, which stops the job remotely once three failed queue items are reported.
Sections you finish are checked off in the contents.