3.4 Immediate and Catch-up Synchronization

The engine described in the previous sections works in “catch up” mode. Synchronization is done in passes, which may be run by a scheduler or launched manually. The synchronization passes use the following logic:

1. Read the target digest

2. Query the delta from the source (target digest is passed as input because the source needs it to compute the delta).

3. Apply the delta to the target (which will resolve conflicts and update its digest at the end).

4. Log success/failure

In a number of scenarios, it is important to propagate changes as soon as possible instead of having to wait for a synchronization pass. This “immediate” mode involves a slightly different process:

1. Detect that a resource has changed on the source side.

2. Prepare a delta that represents the change.

3. Apply the delta to the target.

4. Log success/failure.

The SData synchronization protocol is designed to handle the two modes under a common protocol. Of course, the source implementation will be different in the two modes (in catch-up mode, the source needs to provide an entry point to query the delta, in immediate mode, the source needs to be able to detect changes), but the protocol is designed so that the target side can handle both modes in a uniform way (the target does not need to be aware of which mode is used).

Also, the SData synchronization protocol is designed to support a mix of the two modes. This feature is important because the immediate mode is challenging. For example, one of the applications may be offline or the target application may be down when the source tries to propagate changes. This can be solved by introducing a queue to guarantee delivery but this has drawbacks (queue needs to remain active if source application is closed, queue may end up wasting lots of resources, especially is a target is never reachable, etc.). An alternative is to consider that the immediate mode is only a “best effort” to propagate changes, and to use the “catch up” mode to ensure that systems will be brought back in sync periodically even if some immediate sync requests have been lost.

In immediate mode, the architecture is typically the following:

The main difference with the catch up mode is that the engine does not read the target digest any more (this would involve an extra roundtrip), it only pushes the delta to the target.

The SData protocol handles the immediate mode and the mix of modes by using a “rich” (self-describing) synchronization feed to transfer the changes. The SData synchronization feed contains:

• A batch of actions to create/update/delete resources.

• The synchronization metadata for the individual resources in the batch (endpoint + tick + modification timestamp triplet).

• The source digest.

• A syncMode element to distinguish immediate synchronization from catch up synchronization.

The target provider must check the syncMode element. If it is set to immediate, the target should reject the feed it the tick values of the entries are not contiguous with their corresponding tick value in the target digest. This additional check prevents the target from getting out of sync if immediate synchronization requests are not delivered reliably.

---