Skip to main content

Cross-Simulation Data Sync

Synchronize mock data between simulations with configurable rules, field mapping, and trigger conditions. Enable stateful workflows across multiple mock services that interact and share data.

Prerequisites​

  • Multiple mock simulations in the same workspace
  • Understanding of environments/state management
  • Knowledge of field mapping and template syntax

Understanding Data Sync​

Data sync rules automatically copy data from one simulation to others when events occur.

Use Cases:

  • When order created on Order Service, sync to Notification Service
  • When user updates profile, propagate to all downstream services
  • When payment completes, update status across multiple systems
  • Mirror state between mock services

Components:

  • Source Simulation - Where sync is triggered
  • Trigger - HTTP method + path that activates sync
  • Targets - Destination simulations and collections
  • Field Mappings - Map source data to target schema
  • Execution - Synchronous or asynchronous

Creating Sync Rules​

Access Sync Configuration​

  1. Open source simulation
  2. Click Data Sync or Sync Rules tab
  3. Click Create Sync Rule or New Rule

Define Trigger​

Specify when sync activates:

{
  "name": "Sync Applications",
  "description": "When application submitted, sync to underwriting and notification services",
  "trigger_method": "POST",
  "trigger_path": "/api/applications",
  "sync_mode": "async",
  "retry_on_failure": true,
  "max_retries": 3
}

Trigger Fields:

  • name - Rule name
  • description - What this rule does
  • trigger_method - HTTP method (GET, POST, PUT, PATCH, DELETE)
  • trigger_path - URL path pattern
  • sync_mode - "sync" (blocking) or "async" (non-blocking)
  • retry_on_failure - Retry on target failures
  • max_retries - Number of retry attempts

Define Targets​

Specify where data goes:

{
  "targets": [
    {
      "target_instance_id": "underwriting-service-id",
      "target_collection": "applications",
      "action": "create",
      "field_mappings": {
        "external_id": "{{body.id}}",
        "applicant_name": "{{body.name}}",
        "applicant_email": "{{body.email}}",
        "application_amount": "{{body.amount}}",
        "status": "{{body.status}}"
      }
    },
    {
      "target_instance_id": "notification-service-id",
      "target_collection": "notifications",
      "action": "create",
      "field_mappings": {
        "event_type": "application_submitted",
        "recipient_email": "{{body.email}}",
        "subject": "Application Received",
        "timestamp": "{{now:iso}}"
      }
    }
  ]
}

Target Fields:

  • target_instance_id - UUID of target simulation
  • target_collection - Collection name on target
  • action - "create", "update", or "delete"
  • field_mappings - Map source fields to target schema
  • match_key - Field for finding records (update/delete)
  • match_value_template - Value to match on

Field Mapping Syntax​

Map source request data to target collection fields:

Template Variables:

VariableExampleDescription
{{body.field}}{{body.id}}Request body field
{{body.nested.field}}{{body.user.email}}Nested body field
{{query.param}}{{query.userId}}Query parameter
{{headers.name}}{{headers.authorization}}Request header
{{path.id}}{{path.id}}Path parameter
{{now:iso}}{{now:iso}}Current ISO timestamp
{{now:unix}}{{now:unix}}Current Unix timestamp
{{random:uuid}}{{random:uuid}}Random UUID
{{request.method}}{{request.method}}HTTP method
{{request.path}}{{request.path}}Request path

Examples:

{
  "field_mappings": {
    "id": "{{body.id}}",
    "email": "{{body.user.email}}",
    "userId": "{{path.userId}}",
    "createdAt": "{{now:iso}}",
    "trackingId": "{{headers.x-request-id}}",
    "action": "{{request.method}}"
  }
}

Conditional Sync​

Optional: Add conditions to trigger sync only when criteria met:

{
  "trigger_conditions": {
    "body.status": "approved",
    "body.amount": { "$gt": 1000 }
  }
}

Sync only occurs if request body matches these conditions.

Managing Sync Rules​

List Sync Rules​

View all rules for a simulation:

  1. Open simulation
  2. Click Data Sync tab
  3. See all sync rules with:
    • Rule name
    • Trigger path
    • Target count
    • Enabled status

Get Rule Details​

View full rule configuration:

  1. Click rule name
  2. See complete definition:
    • Trigger settings
    • All targets with field mappings
    • Execution statistics

Update Sync Rule​

Modify rule configuration:

  1. Open rule
  2. Click Edit
  3. Update fields (trigger, targets, mappings)
  4. Click Save

Changes take effect immediately for new executions.

Delete Sync Rule​

Remove a sync rule:

  1. Open rule
  2. Click Delete
  3. Confirm deletion
  4. Rule no longer triggers

Enable/Disable Rule​

Pause without deleting:

  1. Open rule
  2. Toggle Enabled switch
  3. Rule skipped for new requests

Triggering Sync​

Automatic Trigger​

Sync occurs automatically when matching request arrives:

curl -X POST https://instance.surestage.io/api/applications \
  -H "Content-Type: application/json" \
  -d '{
    "id": "app-123",
    "name": "John Doe",
    "email": "john@example.com",
    "amount": 50000,
    "status": "approved"
  }'

If trigger matches, sync rule automatically executes and propagates data to targets.

Sync Modes​

Synchronous (sync):

  • Request waits for all targets to complete
  • If any target fails, entire request fails
  • Slower but guaranteed delivery
  • Use for critical syncs

Asynchronous (async):

  • Request returns immediately
  • Targets sync in background
  • Faster but no delivery guarantee
  • Use for non-critical syncs

Monitoring Sync Executions​

View Execution History​

See all sync executions:

  1. Open sync rule
  2. Click Executions tab
  3. View paginated list:
    • Timestamp
    • Request that triggered sync
    • Status (succeeded, failed, retrying)
    • Target simulations affected
    • Duration

Get Execution Details​

View specific execution:

  1. Click execution in list
  2. See detailed results:
Sync Rule: Sync Applications (async)
Execution ID: exec-abc123
Triggered At: 2026-01-15T10:30:00Z
Trigger: POST /api/applications

Overall Status: PARTIALLY SUCCESSFUL

Targets:
├─ underwriting-service
│  ├─ Status: SUCCESS
│  ├─ Collection: applications
│  ├─ Action: create
│  └─ Duration: 45ms
│
├─ notification-service
│  ├─ Status: FAILED (Retry 1/3)
│  ├─ Error: Connection timeout
│  └─ Next retry: 2026-01-15T10:31:05Z
│
└─ audit-service
   ├─ Status: PENDING
   ├─ Action: create
   └─ Queued for execution

Request Data Synced:
{
  "external_id": "app-123",
  "applicant_name": "John Doe",
  "applicant_email": "john@example.com"
}

Retry Failed Sync​

Retry a failed sync execution:

  1. Open failed execution
  2. Click Retry
  3. Sync attempts again with exponential backoff
  4. Monitor status in execution history

Retries use original source data with updated timestamps.

Sync Health Dashboard​

View overall sync health:

Access Dashboard​

  1. Open simulation
  2. Click Sync Dashboard or Data Sync Health
  3. See summary:
Sync Health Summary

Total Sync Rules: 5
├─ Enabled: 4
└─ Disabled: 1

Executions (24h):
├─ Total: 245
├─ Successful: 238 (97.1%)
├─ Failed: 5 (2.0%)
├─ Retrying: 2 (0.8%)

Recent Failures:
├─ Rule: Sync Applications
│  ├─ Failures: 2
│  ├─ Target: notification-service
│  └─ Error: Connection timeout
│
└─ Rule: Sync Payments
   ├─ Failures: 3
   └─ Target: audit-service

Top Rules by Volume:
├─ Sync Applications: 145 executions
├─ Sync Payments: 78 executions
└─ Sync Users: 22 executions

Field Mapping Examples​

Simple User Sync​

Copy user data to multiple services:

{
  "trigger_method": "POST",
  "trigger_path": "/api/users",
  "targets": [
    {
      "target_instance_id": "profile-service",
      "target_collection": "users",
      "action": "create",
      "field_mappings": {
        "id": "{{body.id}}",
        "name": "{{body.name}}",
        "email": "{{body.email}}",
        "status": "{{body.status}}"
      }
    },
    {
      "target_instance_id": "notification-service",
      "target_collection": "subscribers",
      "action": "create",
      "field_mappings": {
        "user_id": "{{body.id}}",
        "email_address": "{{body.email}}",
        "preferences": "{{body.notification_preferences}}"
      }
    }
  ]
}

Update with Match Key​

Update existing records:

{
  "target_instance_id": "order-service",
  "target_collection": "orders",
  "action": "update",
  "field_mappings": {
    "status": "{{body.status}}",
    "updated_at": "{{now:iso}}"
  },
  "match_key": "order_id",
  "match_value_template": "{{body.id}}"
}

Finds order by ID and updates status.

Nested Field Mapping​

Map nested request data:

{
  "field_mappings": {
    "id": "{{body.transaction.id}}",
    "amount": "{{body.transaction.amount}}",
    "currency": "{{body.transaction.currency}}",
    "payer_name": "{{body.user.profile.full_name}}",
    "payer_email": "{{body.user.contact.email}}"
  }
}

Combining Multiple Sources​

Use request path, query, and body:

{
  "field_mappings": {
    "record_id": "{{path.id}}",
    "account_id": "{{query.accountId}}",
    "data": "{{body.payload}}",
    "version": "{{headers.api-version}}",
    "sync_timestamp": "{{now:iso}}"
  }
}

Sync Workflows​

Multi-Tier Notification​

When order placed, notify multiple services:

Customer Service (trigger)
    ↓
├─ Inventory Service (update stock)
├─ Payment Service (process payment)
├─ Notification Service (send email)
└─ Analytics Service (log event)

Approval Workflow​

Different actions based on approval:

Request Created
    ↓
├─ If approved:
│  ├─ Sync to Fulfillment Service
│  └─ Sync to Accounting Service
│
└─ If rejected:
   ├─ Sync to Audit Service
   └─ Sync to Notification Service (send rejection email)

Use conditional sync rules.

Cascading Updates​

Update propagates through chain:

User Profile Updated
    ↓ (Sync Rule 1)
User Service Updates → Profile Cache Service
    ↓ (Sync Rule 2)
Profile Service Updates → Notification Service

Each update triggers subsequent syncs.

Best Practices​

Design Sync Rules​

  • Keep field mappings simple and clear
  • Document what each mapping represents
  • Use descriptive rule names
  • Include descriptions explaining purpose

Error Handling​

  • Enable retries for critical syncs
  • Use async for non-blocking operations
  • Monitor sync health dashboard
  • Alert on repeated failures

Performance​

  • Minimize target count per rule
  • Use async mode when possible
  • Batch related syncs in one rule
  • Monitor sync duration trends

Testing​

  • Test field mappings with sample data
  • Verify target collections receive correct data
  • Check retry behavior
  • Monitor execution logs

Troubleshooting​

Sync Not Triggering​

  • Verify rule is enabled
  • Check trigger method matches (POST, GET, etc.)
  • Verify trigger path matches request path
  • Check sync conditions if set

Sync Failing​

  • Check target simulation exists and is active
  • Verify target collection exists
  • Review field mapping syntax
  • Check network connectivity

Wrong Data Synced​

  • Verify field mappings are correct
  • Check template variable syntax
  • Test mappings with sample data
  • Review execution details for actual values

Performance Issues​

  • Switch to async mode
  • Reduce number of targets
  • Check target simulation performance
  • Monitor sync duration trends

API Reference​

Sync Rule Management​

MethodPathDescription
GET/instances/:id/sync-rulesList rules
POST/instances/:id/sync-rulesCreate rule
GET/instances/:id/sync-rules/:ruleIdGet rule details
PATCH/instances/:id/sync-rules/:ruleIdUpdate rule
DELETE/instances/:id/sync-rules/:ruleIdDelete rule
POST/instances/:id/sync-rules/:ruleId/enableEnable rule
POST/instances/:id/sync-rules/:ruleId/disableDisable rule

Execution Management​

MethodPathDescription
GET/instances/:id/sync-rules/:ruleId/executionsList executions
POST/sync-executions/:execId/retryRetry execution

Dashboard​

MethodPathDescription
GET/instances/:id/sync-dashboardGet health dashboard

Next Steps​