Data Migration vs. Data Conversion

IMG_0170

Data Migration: Is the transferring of data from one system to another.  Think of this as “data on the move”.

Data Conversion: Is the transformation (or translation) of data from one format to another.  Think of this as “data changing state”.

Data migration is used when upgrading the data model, acquiring a new pipeline of data (additional data feed), or moving from one system or database to another.

Data Conversion is used to extract data from one source, transform it, and prepare for load into another system.

We are working on multiple projects that include massive data conversion and data migration activities.  Millions of records will be converted and migrated, and accuracy is critical.  When the process runs, it will look simple.  But the prep work is key.  Every data element must be reviewed to determine the unique identifiers (what are the keys?), the issues with these identifiers (oh no, were the account numbers reused and no longer unique), the expected landing position, and the meta-data (what is the definition for this data element?).

For people who aren’t involved in the process, I’m sure they wonder what the big deal is.  Take some data – convert it and migrate it.  Aren’t you done yet?  For people who are involved in the process and do it well, they know this can take weeks (maybe months) and every step needs to be perfect.  Failure to understand the current state of the data, failure to validate the business rules for the migration, or failure to test thoroughly can result in corrupt data.  And who wants to receive a bank statement with someone else’s data (name, address, account activity, or balances)?  Okay, maybe you’d like a peak into someone else’s data.  🙂  But I’m betting you don’t want someone else to receive yours!

Now to make it real for anyone not working in the data teams today…

Think of standard chores you do at home.  Taking all the little notes you wrote yourself and consolidating them into one list.  Data conversion – you’re an expert.  Now take that list and move it from paper into your computer.  Data migration.  See how simple it can be?  And rewarding?  And yet, it all takes time.

To all the unsung heroes working on data conversion and data migration projects, I salute you.  You inspire me.  It’s not glamorous work.  It’s not too much fun.  Okay, maybe it is for geeks.  But it sure is important.

 

7 Comment

  1. Anonymous says: Reply

    peek

    1. Anonymous says: Reply

      and back at you!

  2. Mir says: Reply

    Best article ever. Amazing piece..
    I’m sharing this with my team..

    Thank you

    1. Anonymous says: Reply

      Thanks Mir. Let me know if they have any feedback!

  3. Anonymous says: Reply

    To me data related work is the most interesting part.

    1. Anonymous says: Reply

      Thanks! It takes people like you to keep it all moving forward!

  4. 203522 says: Reply

    Hahaha well said! I agree completely!

Leave a Reply