Feed API Documentation

What is this?

Admin note: This is the high-speed “highway” for moving massive amounts of data between your e-shop and other business systems (like your warehouse or accounting software). It is designed to handle thousands or even millions of records efficiently.

Who is this for? Administrators should read this to understand how the platform stays in sync with your other business tools. While the technical “how-to” is for your developers, you should know that this feature ensures your product stock, prices, and orders are always up-to-date across all your systems.

Key benefits for the e-shop:

Rock-Solid Data Accuracy: Automatically compare your entire product catalog and update only what has actually changed, preventing data errors.
Large-Scale Syncing: Effortlessly handle huge files (even gigabytes in size) that would crash standard import tools.
Simplified Logic: Instead of tracking every small change manually, your developers can just send the “full picture” and let our system handle the rest.
Automatic Cleanup: Ensure that products removed from your warehouse system are also automatically hidden or removed from your e-shop.

What is Feed API?

When synchronizing two systems, you have two primary approaches:

card: Classical API
icon: refresh-cw
Read data via API, detect changes locally, and push only modifications. Requires change tracking and state management on the client side.
**Best for:**
- Real-time synchronization
- Frequent small updates
- Fine-grained control over changes
card: Feed API
icon: zap
Send one large JSON (even gigabytes) with all values. The system efficiently compares and updates only real changes.
**Best for:**
- Mass data synchronization
- Periodic full syncs
- Simpler integration logic

How Feed API Works

1. Efficient Preload

The system efficiently preloads existing data for comparison using optimized database queries.

2. Intelligent Matching

Records are matched using lookups (code:, id:) or unique identifiers. Even items without unique IDs (like line items) are matched using a probability mechanism.

3. Change Detection

The system compares all data and identifies actual changes, eliminating false differences.

4. Selective Updates

Only real changes are written to the database, minimizing overhead and maintaining audit trails.

Core Components

Feed API leverages two powerful tools: Mapper and Importer

card: Mapper
icon: settings
Transforms external data formats (XML, JSON, CSV) into the internal format. Handles Mapper transformations, data transformations, and type conversions.
**Key Features:**
- JSONPath and XPath support
- Expression evaluation
- Lookup resolution
- Nested entity mapping
[navigate:mapper.md|View Docs|book-open]
card: Importer
icon: upload
Processes mapped data and intelligently updates the database. Handles upserts, deletions, and relationship management.
**Key Features:**
- Upsert operations
- Lookup-based matching
- Probabilistic matching
- Batch processing
[navigate:imports.md|View Docs|book-open]

Key Directives

Essential directives for Feed API operations

Lookups (code:, id:, etc.)

Identify records using various lookup types instead of database IDs.

"category": "code:ELECTRONICS"
"manufacturer": "id:12345"
"customer": "email:john@example.com"

@removeAll

Controls whether existing items not in the feed should be removed.

"items": [
  { "code": "ITEM-1", "quantity": 10 }
],
"items@removeAll": true  // Remove items not in feed

Default: true for array/object mode, false for merge mode

@operation

Explicitly control the operation type for each record.

{ "@operation": "upsert", "code": "PROD-1", "name": "Product 1" }
{ "@operation": "delete", "code": "PROD-OLD" }
{ "@operation": "create", "name": "New Product" }

Values: create, update, upsert, delete

@where

Mass operations using where clauses for mass updates or deletions.

{
  "@operation": "update",
  "@where": "category = 'code:OLD_CAT'",
  "category": "code:NEW_CAT"
}

@atomic

Controls transaction behavior and error handling.

"@atomic": "batch"  // Recommended for large feeds
"@atomic": true     // All-or-nothing
"@atomic": false    // Independent records

Example: Products Feed

Complete example of synchronizing products via Feed API

1. Source Data (External System)

{
  "products": [
    {
      "productCode": "WIDGET-001",
      "productName": "Super Widget",
      "price": 99.99,
      "categoryCode": "ELECTRONICS",
      "variants": [
        {
          "sku": "WIDGET-001-RED",
          "color": "Red",
          "stock": 50
        },
        {
          "sku": "WIDGET-001-BLUE",
          "color": "Blue",
          "stock": 30
        }
      ]
    },
    {
      "productCode": "GADGET-002",
      "productName": "Mega Gadget",
      "price": 149.99,
      "categoryCode": "ELECTRONICS"
    }
  ]
}

2. Mapper Configuration

type JSON

@atomic = "batch"
@language = "en"

products = '$.products[*]' {
    id[code] = 'productCode'
    name = 'productName'
    price = 'price'
    category[code] = 'categoryCode'
    
    variants = 'variants[*]' {
        code = 'sku'
        color = 'color'
        stockQuantity = 'stock'
        @removeAll = "true"
    }
}

3. Resulting Import JSON

{
  "@atomic": "batch",
  "products": [
    {
      "id": "code:WIDGET-001",
      "name": "Super Widget",
      "price": 99.99,
      "category": "code:ELECTRONICS",
      "variants": [
        {
          "code": "WIDGET-001-RED",
          "color": "Red",
          "stockQuantity": 50
        },
        {
          "code": "WIDGET-001-BLUE",
          "color": "Blue",
          "stockQuantity": 30
        }
      ],
      "variants@removeAll": true
    },
    {
      "id": "code:GADGET-002",
      "name": "Mega Gadget",
      "price": 149.99,
      "category": "code:ELECTRONICS"
    }
  ]
}

**What Happens:**
- Products are matched by code (`code:WIDGET-001`)
- Category is resolved via lookup (`code:ELECTRONICS`)
- Existing variants not in the feed are removed (`@removeAll: true`)
- New variants are created, existing ones are updated
- Only actual changes are written to database
- Processing is done in batches for optimal performance

Example: Invoices Feed

Synchronizing invoices with line items

{
  "@atomic": "batch",
  "invoices": [
    {
      "id": "code:INV-2024-001",
      "customer": "code:CUSTOMER-123",
      "issueDate": "2024-01-15",
      "dueDate": "2024-02-15",
      "items": [
        {
          "product": "code:WIDGET-001",
          "quantity": 5,
          "unitPrice": 99.99,
          "description": "Super Widget - Red"
        },
        {
          "product": "code:GADGET-002",
          "quantity": 2,
          "unitPrice": 149.99,
          "description": "Mega Gadget"
        }
      ],
      "items@removeAll": true
    }
  ]
}

**Probabilistic Matching:**
Even though invoice line items typically don't have unique identifiers, the system uses a probability mechanism to match them based on product, quantity, and other fields. This ensures accurate updates without duplicate line items.

Generating Feeds

Feed API can also generate data exports in various formats

You can use the Feed API not just for imports, but also to generate data exports:

Query-Based Export

Use the query language to extract data and export it in various formats.

GET /api/products?format=json&export=true

Reverse Mapper

Use mappers in reverse to transform internal data back to external formats.

Forward Mapper

Apply forward mappers to generate specialized export formats.

Supported Export Formats:

JSON, XML, CSV, Excel, PDF, HTML, Custom

Best Practices

Use batch mode for large feeds — Set @atomic: "batch" for optimal performance with large datasets.
Use lookups instead of database IDs — Prefer code:, email:, or other business keys for better portability.
Leverage @removeAll for complete synchronization — Use @removeAll: true to ensure deleted items are removed from the target system.
Test with small datasets first — Validate your mapper and import logic with a small subset before processing large feeds.
Monitor import results — Check import logs and error reports to identify and fix data issues.