Approach B — Smart Gateway

AMS Data Integration
Scope of Work

What Digicon builds, what the partner builds, and why we need them.

Decisions Already Made

Data Scope
All categories, phased
Partner Role
Systems integrator / dev agency
Platform Strategy
Platform-agnostic
Integration Model
Hybrid (push + pull)
Phase 1 Goal
Full onboarding automation
Partner Visibility
API spec only — no internal access
Commercial Model
Not decided — design for flexibility
Architecture
Approach B — Smart Gateway
1

How the Smart Gateway Works

The partner extracts data from client platforms and pushes it to Digicon's API. Digicon's transformation engine normalizes, validates, and routes the data into AMS internals. Neither side touches the other's systems directly.

HubSpot Salesforce Shopify WooCommerce Custom CRM
Partner: Platform Connectors
↓   JSON via REST API
Digicon: Smart Gateway API
↓   Transform & Validate
Digicon: Transformation Engine
↓   Normalized Data
AMS (Airtable + Drive + Config)

Why This Split?

Digicon keeps control of what matters most: the data model, quality rules, and transformation logic. This is core IP. If the data enters AMS wrong, every downstream system (scripts, video pipeline, analytics) breaks.

The partner does what they're good at: building connectors to dozens of platforms, managing OAuth flows, handling API rate limits, and extracting data reliably. This is commodity engineering — valuable but not differentiating for Digicon.

2

Digicon Builds: The Smart Gateway

Everything from the API endpoint inward. Digicon owns the contract (schema), the intelligence (transformation), and the destination (AMS internals).

Digicon

2A. Ingest API

  • REST API endpoints for each data type POST /ingest/products, /ingest/assets, /ingest/campaigns, /ingest/audience
  • Webhook receiver for real-time events Product updated, new asset uploaded, campaign status changed
  • Bulk import endpoint for initial sync Accept batches of 1,000+ records, async processing with job status
  • Request validation and error responses Schema validation, field-level error messages, idempotency keys
  • Rate limiting per API key Configurable per-client limits, retry-after headers
Digicon

2B. Auth & Multi-Tenancy

  • API key generation and management Per-client keys, revocation, rotation support
  • Client isolation at the API layer Every request scoped to a client — no cross-tenant data leaks
  • Usage tracking and metering API calls, records processed, storage used — per client, per day
  • Audit logging Who called what, when, with what result — retained 90 days minimum
Digicon

2C. Transformation Engine

  • Standard data schemas (the "contract") Product, Asset, Campaign, Audience — published as JSON Schema + docs
  • Platform-specific field mapping rules e.g., HubSpot "deal_name" → AMS "product_name". Configurable per platform.
  • Data normalization and cleaning Date format normalization, currency conversion, text sanitization
  • Validation rules engine Required fields, value ranges, cross-field dependencies, custom rules per client
  • Conflict resolution logic How to handle duplicates, merge conflicts, stale data overwrites
  • Error quarantine and retry queue Failed records held for review, not silently dropped
Digicon

2D. Auto-Provisioning

  • Airtable base creation for new clients Standard table structure, fields, views — spun up from template
  • Google Drive folder structure Client folder, asset subfolders, permission assignment
  • AMS config generation use_for key, server config, pipeline settings — auto-generated on first data push
  • Onboarding status tracking Track: connected → first sync → validated → production-ready
Digicon

2E. AMS Write Layer

  • Airtable record creation and updates Batch writes with conflict detection, field-level change tracking
  • Google Drive asset uploads Files placed in correct client folder, metadata attached
  • Change event emission Webhook out: "new product synced", "asset updated" — for AMS pipeline triggers
  • Sync state management Track last sync timestamp, delta detection, full re-sync capability
Digicon

2F. Developer Portal & Docs

  • Published API specification (OpenAPI / Swagger) Full endpoint docs, request/response examples, error codes
  • Standard data schemas with field descriptions What each field means, which are required, allowed values
  • Sandbox environment for partner testing Test API keys, mock data, no production side effects
  • Integration guide and quickstart Step-by-step: get key → push first record → verify in sandbox
3

Partner Builds: Platform Connectors

Everything from the client's platform to the Digicon API. The partner owns extraction, connection management, and delivery orchestration. They never touch AMS internals.

Partner

3A. Platform Connectors

  • Data extraction modules per platform HubSpot, Salesforce, Shopify, WooCommerce — Phase 1 targets
  • API integration with each platform's SDK Handle pagination, rate limits, retry logic, API versioning
  • Data extraction for each AMS data type Products/catalog, assets/media, campaigns, audience/contacts
  • Incremental sync (delta detection) Only extract changed records since last sync, not full table each time
  • Platform-specific webhook listeners Subscribe to real-time events: product updated, new order, etc.
Partner

3B. OAuth & Connection Management

  • OAuth flow implementation per platform Client clicks "Connect HubSpot" → OAuth → tokens stored securely
  • Token refresh and expiry handling Auto-refresh before expiry, graceful re-auth when refresh fails
  • Connection health monitoring Detect stale/revoked connections, alert Digicon API
  • Credential storage (encrypted) Partner stores client credentials — Digicon never sees platform tokens
Partner

3C. Push & Pull Orchestration

  • Scheduled pull jobs (bulk syncs) Daily/hourly full or incremental syncs — configurable per client
  • Real-time event forwarding (push) Platform webhook → transform to Digicon schema → POST to Digicon API
  • Job queue and retry logic Failed pushes retried with exponential backoff, dead-letter after N failures
  • Data formatting to Digicon's standard schema Map platform fields to Digicon's published schema before pushing
  • Sync status reporting Report sync results to Digicon API: records sent, errors, timestamps
Partner

3D. Connection Management UI

  • Client-facing connection interface Simple UI: "Connect your HubSpot" → OAuth → "Connected ✓"
  • Connection status dashboard Show: connected platforms, last sync time, sync health, error count
  • Field mapping UI (optional, for custom fields) Let client map their custom fields to Digicon standard fields
  • Disconnect / reconnect flow Clean disconnect, token revocation, re-auth flow
4

Shared Responsibilities

These items sit at the boundary between Digicon and the partner. Both sides must collaborate on them, with clear ownership of each sub-task.

Shared

Integration Contract & Testing

  • API schema co-design (initial) Digicon leads. Publishes schema. Partner reviews for feasibility against platform APIs.
  • End-to-end integration testing Both. Partner pushes test data → Digicon verifies it arrives correctly in AMS.
  • Error handling protocol Both. Define: what errors does Digicon return? How does partner retry? When to escalate?
  • New platform onboarding process Partner leads. Builds connector. Digicon supports: adds mapping rules, tests with sandbox.
  • Monitoring and alerting for sync health Both. Partner monitors extraction. Digicon monitors ingestion. Shared dashboard.
  • Security review and data handling agreement Both. Define: what data is transmitted, how it's encrypted, retention policies, compliance.
5

Why We Need the Partner

What happens if Digicon tries to build everything in-house — and why the partner accelerates us.

Component Without Partner With Partner Why It Matters
Platform connectors
(HubSpot, Salesforce, Shopify, etc.)
Digicon builds each connector from scratch. Each platform has unique APIs, auth flows, pagination, and rate limits.
Digicon: 3-4 weeks per platform
Partner has existing connector frameworks or iPaaS experience. Builds connectors in parallel.
Partner: 1-2 weeks per platform
With 4 Phase 1 platforms, the partner saves 8-12 weeks of Digicon engineering time.
OAuth management
(Token storage, refresh, revocation)
Digicon must build secure OAuth for each platform — each with different grant types, scopes, and refresh mechanisms. Partner specializes in this. Handles credential lifecycle. Digicon never touches client platform tokens. Security separation. Digicon never stores third-party credentials. Reduces attack surface.
Connection UI
(Client-facing connect/disconnect)
Digicon builds a client-facing frontend — outside current core competency (backend + AI pipeline). Partner builds the UI. It can be whitelabeled or embedded into Digicon's onboarding flow. Digicon stays focused on the transformation engine and AMS pipeline — the actual IP.
Platform expertise
(API quirks, edge cases, rate limits)
Learning curve for every new platform. Undocumented behaviors, pagination bugs, breaking API changes. Partner already knows these platforms. Has handled edge cases before. Risk reduction. Fewer surprises in production. Faster debugging.
Scaling to new platforms
(Phase 2, 3, and beyond)
Each new platform is a new project. Digicon's product team is bottlenecked. Partner adds platforms without blocking Digicon. "Add Mailchimp support" is a partner work order, not a Digicon sprint item. Scalability. Digicon grows platform coverage without growing headcount.

The core risk of NOT having a partner: Digicon spends 60-70% of engineering time on connector plumbing instead of the transformation engine, auto-provisioning, and AI pipeline — which is where the product value lives. The partner handles the undifferentiated heavy lifting so Digicon can focus on its actual competitive advantage.

6

Phase 1 Delivery Checklist

What "done" looks like for Phase 1: a new client connects their platform, and AMS self-configures with their product catalog data.

Digicon Delivers

  • Ingest API — products endpoint live
  • Auth system — API keys, multi-tenancy
  • Standard Product schema published
  • Transformation engine — field mapping for Phase 1 platforms
  • Validation rules for product data
  • Auto-provisioning — Airtable base + Drive folders
  • AMS write layer — Airtable + config generation
  • Sandbox environment for partner testing
  • API docs (OpenAPI spec + integration guide)
  • Usage tracking and audit logging
  • Error quarantine and monitoring dashboard

Partner Delivers

  • HubSpot connector — product extraction
  • Salesforce connector — product extraction
  • Shopify connector — product catalog extraction
  • WooCommerce connector — product catalog extraction
  • OAuth flow for each platform
  • Credential storage (encrypted)
  • Scheduled sync jobs (daily incremental)
  • Real-time webhook forwarding
  • Retry logic and dead-letter handling
  • Connection UI (connect / status / disconnect)
  • Sync status reporting to Digicon API

Phase 1 Success Criteria

A new client connects their HubSpot (or Salesforce, Shopify, WooCommerce) account. Within minutes, their product catalog is synced into AMS. An Airtable base is provisioned, Drive folders are created, and the AMS pipeline is ready to generate content for that client — with zero manual setup by Digicon.

7

What We Need From the Partner

Minimum requirements when evaluating partner candidates.

Technical Requirements

Must Have

  • Production experience with HubSpot, Salesforce, or Shopify APIs
  • Built OAuth 2.0 flows for multi-tenant SaaS before
  • Experience with webhook infrastructure at scale
  • Secure credential storage (encrypted at rest + in transit)
  • REST API integration experience (consuming + building)
  • CI/CD pipeline and automated testing
Operational Requirements

Must Have

  • Ability to work from API spec (no access to Digicon internals)
  • Dedicated point of contact for integration questions
  • SLA for bug fixes on connectors (e.g., 24h response, 72h fix)
  • Willingness to sign NDA and data handling agreement
  • Experience supporting multiple clients on shared infrastructure
  • Transparent pricing (per-connector or per-platform, not per-API-call)
Nice to Have

Preferred

  • Pre-built connector framework or iPaaS experience (n8n, Workato, etc.)
  • Experience with Airtable API (for future collaboration)
  • White-label UI capability (embed connection UI in Digicon's app)
  • SOC 2 compliance or equivalent security certification
  • Experience working with AI/content pipeline companies
Digicon Provides to Partner

What We Give Them

  • Published API spec (OpenAPI/Swagger)
  • Standard data schemas (JSON Schema + docs)
  • Sandbox environment with test API keys
  • Integration guide and quickstart tutorial
  • Dedicated Digicon technical contact
  • Slack channel for real-time support during build