Content (Data, Images, Files) Migration and Ongoing Management

Overview

This article provides details on the extra content management work that is required during a Classic-to-Current Migration project.  It covers the following topics:

  • The Overlap Period (You Have Two Platforms)
  • What Is “Content”?
  • How The Overlap Period Impacts Content Management
  • A Diagram of Content Work By Migration Stage
  • Data (Migrated & ERP-Synced) & Ongoing Management
    • Migrated Data
    • ERP Synced Data (for Standard Connectors & 3rd Party Connectors)
    • Dual-Managed Data
  • Images/Files Migrated & Ongoing Management
    • Migrated Images / Files
    • Dual-Managed Images / Files
    • A Look Under The Hood (How The Images / Files Work Now)

The Overlap Period (You Have Two Platforms)

The migration process involves an overlap period where you have two (2) separate platforms, each with their own separate databases and data, running at the same time.  These are:

  1. Your current live “classic” platform, and
  2. The newly deployed CIMcloud platform that will replace it

For a better understanding of what happens during a migration project, reference these two articles:

What Is “Content”?

For the purpose of this section, the term “content” is used broadly to include virtually all data, images, and files.  This includes all product data, customer data, order / invoice / payment / shipment data, navigation links, web page content, all types of settings data (shipping methods, tax settings, payment processing settings, etc), images (used for products, content, slide shows, etc), digital files (like pdfs, word or excel docs, etc), etc.

How The Overlap Period Impacts Content Management

Because of this overlap period, there is a deliberate content management strategy that has to take place during that overlap period.

It is important to understand the content work you will have to do during this overlap period vs. the content work that will be done for you.  The following are the major buckets that content work falls into.

  • Bucket #1: Data automatically synced from your ERP system
  • Bucket #2: Data copied (from your classic platform to your new platform) with Data Copy #1
  • Bucket #3: Data copied (from classic to new) with Data Copy #2 and Data Copy #3
  • Bucket #4: Images/Files copied (from classic to new) with the Image/File Copy #1 (this is the ONLY Image/File copy)
  • Bucket #5: Content (Data/Images/Files) that you have to dual-manage (in both platforms) until go live
  • Bucket #6: Content that you have to prepare and load in the new platform during the implementation stage
  • Bucket #7: Content in your classic platform that is (intentionally) not copied (sessions, web orders, etc)

A Diagram of Content Work By Migration Stage

This is an overview diagram that provides a visual representation of the “content” work that happens before, during, and after a migration project.

It is broken into 3 sections:

  1. Manually Loaded Data
  2. Manually Loaded Images / Files
  3. ERP Synced Data.

The stages listed are based on the overall migration process stages and steps.

[click the above image to enlarge]

Data (Migrated & ERP-Synced) & Ongoing Management

CIMcloud provides data migration services as part of the standard migration. The goal of this work is to dramatically reduce the amount of data work you have to do to get the new platform ready to test and go live (but there IS still work you have to do).  The purpose of this section is to help you understand how data is seeded in the new version of the platform (during stage 1: getting connected), and how it is managed throughout the implementation and live roll-out stages of the project.

Migrated Data

“Migrated Data” is data that is copied from your existing classic platform database to your new CIMcloud platform database.  CIMcloud uses automated scripts to copy this data as it currently exists.  There is no material content manipulation or changes designed into this system.  It does not cover 100% of your data / content needs, but it does get you way down the road in the implementation of the new CIMcloud platform.

There are three (3) data copies, each with different sets of data, included with your migration project.  All three data copies involve copying data FROM your existing classic platform TO your new CIMcloud platform.  There are no data copy services that move data from the new platform to your existing classic platform.  Additional data copies or custom data copies can be provided by CIM Pro Services for additional pricing.

  • Data Copy #1 (initial bulk copy) – this is a wide-reach initial seeding of data that is programmatically copied during the initial deployment of the new CIMcloud platform (in Stage 1 > Step 3).
  • Data Copy #2 (customer wallet data since copy #1) – this is a small update of a few tables (in Stage 2 > Step 4) intended to copy the new data records that your customers entered/created since Data Copy #1.  It typically includes new: favorites lists, saved credit cards, customer contacts, shipper accounts, and (if you have an anonymous / retail store) anonymous accounts & addresses including guests.
  • Data Copy #3 (customer wallet data since copy #2) – this is another small update (in Stage 3 > Live Cutover) intended to copy the new data records that your customers entered/created since Data Copy #2.  It includes the same data tables copied in Data Copy #2.
    • Note: If completing a soft launch, this data copy might require CIM Pro Services work to create custom migration scripts based on data changes during the soft launch period.

ERP Synced Data

“ERP Sync’d Data” is the data that lives in your ERP / Accounting system and automatically syncs to the CIMcloud platform when it is added or edited (note: deletes are also typically synced).  Once the CIMcloud ERP Driver Sync Tool is installed and connected with your new CIMcloud platform (during the Stage 1 Getting Connected > Step 4 Platform Deployment), the sync tool will keep it up to date.

The following describes what typically happens with ERP Synced Data.

For Standard ERP Connectors (like Sage 100)

  • CIMcloud uses the new, standard ERP sync configurations and data mappings
    • This means you get the latest version of each data sync configuration
    • If you have any UDFs, custom logic, or other customizations that exist in your current classic platform’s ERP sync configs, those will NOT be installed by default.  If you want them maintained, they will need to be addressed by CIM Pro Services via a paid engagement.  CIMcloud does not customize or modify our standard ERP sync configs as part of our standard migration process.
  • CIMcloud initially copies most of the ERP synced data from your classic platform to the new platform and THEN starts syncing to maintain updates.  A new full refresh of all data tables (from the ERP to the new CIMcloud platform) is typically NOT completed.  This strategy is used for:
    • Customer-related data (like customer accounts and addresses)
    • Product-related data (like products, warehouses, inventory levels, aliases, and pricing)
    • Sales Tax related data (like tax schedules, codes, classes, and rates)
    • Various other tables (based on the add-on bundles in scope)
  • CIMcloud does a clean (from scratch) refresh of transaction history data (orders, shipments, invoices, payments, credits, etc) directly from your ERP.
    • CIMcloud does NOT copy this data from your classic platform at all.
    • The volume of history records seeded from your ERP depends on your scope.
      • By default, we bring over 1 year of history + all open records (i.e. open invoices)
      • If you purchased the Advanced B2B Portal Bundle [B], we bring over 3 years of history
  • During the Overlap Period, data will sync from your ERP system to BOTH platforms
    • Our ERP sync tool has two separate “installs” running during this period
    • The original “install” syncs to your existing classic platform (as it always has)
    • The new “install” syncs to the new CIMcloud platform
    • Both “installs” automatically detect and send incremental data changes from your ERP system to their respective platform databases
  • For clarification: Images and Files are not synced from your local servers to CIMcloud via any standard setup we have (only the data from your ERP system is synced).
    • Unless you have a highly unusual custom setup on your classic platform, images and files have never been synced from your servers to CIMcloud.

For 3rd Party Connectors (like Sage 300, 500, and X3; Dynamics GP and AX)

  • CIM Pro Services handles the ERP Sync configurations and sync tool install as part of a separate services engagement billed hourly.
    • CIM Pro Services will follow the below typical guidelines, but may vary based on your specific circumstances.
  • CIM Pro Services copies your existing ERP sync configurations and data mappings
    • They may update them if latest versions are available and provide benefits to your install.
    • This allows your existing customized data mappings to be ported over (but other custom migrations would still be required to make the custom data mappings work on the new platform as they do on your current classic platform).
  • CIM Pro Services copies the ERP sync’d data from your classic platform to the new platform and THEN starts syncing it to maintain updates.  A new full refresh of all data tables (from the ERP to the new CIMcloud platform) is typically NOT completed.  This strategy is used for:
    • Customer related data (like customer accounts and addresses)
    • Product related data (like products, warehouses, inventory levels, aliases, and pricing)
    • Sales Tax related data (like tax scheduled, codes, classes, and rates)
    • Transaction History data (orders, shipments, invoices, payments, credits, etc)
    • Various other tables (based on the add-on bundles in scope)
  • During the Overlap Period, data will sync from your ERP system to BOTH platforms
    • Our ERP sync tool has two separate “installs” running during this period
    • The original “install” syncs to your existing classic platform (as it always has)
    • The new “install” syncs to the new CIMcloud platform
    • Both “installs” automatically detect and send incremental data changes from your ERP system to their respective platform databases
  • For clarification: Images and Files are not synced from your local servers to CIMcloud via any standard setup we have (only the data from your ERP system is synced).
    • Unless you have a highly unusual custom setup on your classic platform, images and files have never been synced from your servers to CIMcloud.

Dual-Managed Data

“Dual-Managed Data” is the data that you will manage on BOTH platforms during the overlap period.  Other than data records (or specific fields on products), this is most or all of the data that is managed in the new platform’s Content Workspace, Product Workspace, and Settings Workspace.  You can think of this data in three (3) primary workflows.

  • Workflow #1 – Data In Both, That You Want The Same In Both
    • We copy all or most of this data for you in Data Copy #1 to initially seed the new platform for you.
    • After Data Copy #1, you will need to prepare and load (via web forms or excel / CSV uploads) any changes to this data (adds, edits, or deletes) to BOTH platforms during the entire overlap period.
    • Note: There may be some minor differences in data formats and destination tables between your classic platform and new CIMcloud platform.  You will need to account for these when loading data.
  • Workflow #2 – Data In Both, That You Want To Change In the New Platform
    • This would typically be data that you want to “reengineer” during your migration project.
    • This is part of the “extra work” that your project manager and plan should account for.
    • We copy all or most of this data for you in Data Copy #1 to initially seed the new platform for you (it will be a copy of what you have in your current classic platform).
    • After Data Copy #1, you can decide which tables / fields you want to change and start making the changes in the new platform only.  Reminder: The new platform is completely separate from your existing classic platform (it runs on it’s own separate database and copy of the data), so you can freely edit the data without impacting your existing classic site.
  • Workflow #3: Data Only In The New Platform
    • This is data that only exists in the new platform.  This is typically part of new features or enhancements that come with the latest version of CIMcloud.
    • To run the new platform some of this data is required (and will need to be configured and tested during Stage 2 Implementation).  An example of this is Worker Groups, Worker Logins, and Features & Settings configuration data (previously unavailable to you to control / make changes).  Other data is optional, and only needs to be prepared and loaded if you are going to use the new feature.

Images/Files Migrated & Ongoing Management

CIMcloud also provides image/file migration services as part of the standard migration (with the same goal as stated above).  We initially seed the new platform with your images and files, but there is still work you have to do during the implementation and live roll-out stages.

Migrated Images / Files

“Migrated Images / Files” are the digital images and files that are copied from your existing classic platform’s file system to the file system on your new CIMcloud platform.  There is no material image or file manipulation or changes designed into this system.  It simply moves a 1-time copy of your images and files.  We reference this as Image / File Copy #1.

Dual-Managed Images / Files

After Image / File Copy #1, you will need to prepare and upload all new (and modified) images and files to BOTH platforms during the entire Overlap Period.

A Look Under The Hood (How The Images / Files Work Now)

For Reference, CIMcloud’s latest platform uses an Amazon S3 “bucket” (part of their Content Delivery Network or CDN) for all image and file storage and delivery.  CIMcloud uses automated scripts to copy your files into this CDN and drop “pointers” or references to these images and files into the CIMclould platform.  The CIMcloud platform interfaces allow you to search for and select images (in various places in the CIMcloud Worker Portal).  These interfaces are actually searching records in a database table inside CIMcloud’s database (populated from the Amazon CDN) that “point” to the images and files on the CDN servers.  This change has some material short-term and long term benefits associated with it, but did result in a few short-term feature take-aways.  Our product R&D team is actively working to close these gaps.

 

Was this article helpful?

Related Articles