Types of Software Integrations

Overview

This article introduces two general types of software integrations / integration projects.  These are just made up classifications to help you distinguish the level of risk & complexity of the project you are considering (or already doing).  It will also help you determine whether or not the Foundation Concepts That Apply to Complex Integration Project will apply to your project.  It’s basically used to help you understand what best practices content your team really needs to pay attention to.

This article covers the following topics:

• Simple Integrations
  • Example Simple Integrations
• Complex Integrations
  • Example Syncing Techniques
  • System of Record (SOR) & Sync Paths

Part of a Series of Articles

The article you are currently reading is part of a series of articles that are all organized and summarized in this Integration Reference Guide (Using the CIMcloud API).

Audience (for This Article)

The intended audience for this article is CIMcloud customers (and their contractors / partners) that are considering building and/or managing / supporting an integration between CIMcloud and any other software system.

Type 1: Simple Integrations

We are using the term “Simple Integration” to describe any integration between two software applications where the data only lives / is stored in only one of the applications (we will call this the SOR or “System of Record” for that data).

Some examples of simple integrations include:

• Web Traffic Analytics Tracking – where a Javascript file is includes in a public website’s template but all of the data is actually collected in the analytics platform through the code that runs in that Javascript file.
  • This typically would not involve / require the CIMcloud API.
• Email Signup Form – this might include a form presented on one site that actually posts to a separate platform (i.e. a form that is included on your public site but posts to Mail Chimp, Hubspot, etc).
  • This typically would not involve / require the CIMcloud API.
• Showing Inventory Quantities – this may occur if you have a retail ecommerce site or CRM system that you want to show inventory availability on by calling the CIMcloud API, passing one or more product skus, and just getting the quantity available back.  In this case, the other application is going to call the API, receives the data, display the data, and then discard it (because all subsequent calls for the data will just hit the CIMcloud API again).
  • This would involve / require the CIMcloud API but is a relatively simple integration.

Type 2: Complex Integrations

We are using the term “Complete Integration” to describe any integration between two software applications where the same data lives / is stored in both of the applications.

Example Syncing Techniques

In these complex integrations, there is some form of data synching that occurs between the two software platforms.  The data syncing my occur using one of more different techniques, but all of them involve some sort of software code to capture and send adds, edits, and deletes (we will collectively call these data changes”) of data records from one software application to the other.  Some techniques include:

• Forms (Code) That Update Both Applications – This occurs when a form (i.e. a form that adds or edits customer information) updates both software applications every time the form is posted.  The form would be housed in one of the applications, and, when posted, would 1) update the database of the application it is in, and 2) call an API on the other application to post to it’s database at the same time.  Note: This can be a problematic technique if data changes occur outside of that form (i.e. through CSV uploads, other applications hitting the database directly,
• Sync Tools or Mechanisms (Triggered or Runs on Timer) – This occurs when a mechanism (software) moves data between the two applications.  There are two general things that have to happen
  • #1: The data change has to be logged / known / queued in some way
    • This can be accomplished in many ways (through database triggers, add / modify date-time fields, dirty flag fields, etc).
    • Alternatively (not desired on large tables of data), if changes can not be reliably detected, the entire table of data can just be sent over and over again (and the destination application gets overridden each time it is sent).
  • #2: The changed data has to be moved to the other application
    • This can be fired by the change event (i.e. the change drops a record in a message queue that gets picked up) or by a tool that runs on a timer (i.e. looks for changed record every 5 minutes and sends them).
    • This can be accomplished through native tools within the source application (i.e. web hooks), in-house / custom build sync tools, database replication tools, commercially available sync tools, or integration as a service providers (that have cloud-based integration platforms to do the syncing)

System of Record (SOR) & Sync Paths

When designing a complex integration strategy, it is important to determine which system will serve as the “System of Record” or SOR.  Typically only one of the applications is the SOR or “System of Record” for the data (that is the application that is considered to have the master copy / ultimate source of truth for the data).   Note: The SOR for a record of data can be split between two systems by field.  For example, your ERP my be the SOR for a product sku, name, and price, but your ecommerce platform is the SOR for the product’s description and images.  In this case, there are two SORs on the products table but only one SOR for each of the fields on the products table.  Caution: If you have two or more applications that are the SOR for the same field(s) of data (on any table), it becomes significantly more complex to reliably execute the sync.

The following “Sync Paths” can be considered when designing a complex integration project.  Note: There are other variations (and combinations of these that can occur).  These are presented to assist with the Solution Architect’s and Technical Planner’s jobs.

• 1-Way Sync (One SOR)
  • This occurs when all data changes occur in one application and are copied / synced to the other application for us.
  • Example: All customer accounts are only added / edited in your CRM system, and a sync is set up to start and continue copying that account data into your ERP system (starts when the first order is placed and then continues for all changes to that account).  In this example, your sales / customers service reps would know to only edit customer account information in the CRM (and then it would flow to your ERP system).
• 1-Way Sync (SOR Handoff)
  • This occurs when data moves from 1 platform to the other and the SOR responsibility is passes on the one and only sync of the data.
  • Example: An order is placed on your ecommerce site and then synced into your ERP system for fulfillment.  As soon as the order syncs in, the SOR moves from the commerce platform to the ERP system. At that point NO changes are made to the order in the ecommerce system (only changes can occur in the ERP system).
• 2-Way Sync (One SOR Per Field)
  • This occurs when the same data record moves both directions (between two platforms) but changes to each field in the record only occur in one of the platforms.
  • Example: All changes to product sku, name, and price occur in your ERP and sync to your Ecommerce platform’s product catalog.  All of the “rich product catalog data” including product category assignments, related product assignments, full descriptions, spec data, images, etc occur in the ecommerce platform’s admin tools, but you want just the description to flow back to the ERP system.  In this case, your product data is a two way sync, but each field only flows a max of 1 direction (sku, name, and price go from ERP to ecommerce, description goes from ecommerce to ERP, all other fields only live / stay in the one application that are in).
  • Recommendation: Attempt to avoid this sync path if practical.
• 2-Way Sync (Two SOR’s Per Field)
  • This occurs when the same field of data, on the same record, can be updated in either application, and the update needs to sync to the other application. This can be very complex to execute because concurrent changes have to be reconciled (if an edit occurs and nearly the same time, to the same record, in both systems… which one wins?)
  • Example: If the company name and address on a customer can be edited in your CRM and in your ERP, and they share the data, the sync will need to go in both directions.
  • Recommendation: Work hard to avoid this sync path.  If you can avoid this sync path without significant business loss, avoid it and keep the SOR for each field of data (and all data changes to a single application).