No matter how carefully you prepare and attempt to future-proof your setup, you can’t realistically keep a business running on the same system indefinitely. Sooner or later the industry around you will change to the extent that the underlying architecture can’t keep up, you’ll want to move to a new system, and you’ll need to face a full data migration.
I describe it as intimidating because it should be: if you think it’s no cause for concern, then you understand neither its significance nor the threat it poses. While it can be perfectly smooth in ideal circumstances, the complexities of the real world tend to get in the way, making it something that you’ll narrowly scramble through in the best-case scenario.
Why is this, when even a messy data migration is something you can muddle through easily enough? Well, it comes down to the long-term effects you can encounter if you don’t go about things the right way. Let’s take a closer look at some of them:
Sometimes Needing to Manually Map Fields
When you move your data over to a new system, there’s a strong chance the fields won’t be identical. Perhaps you’ll have added new fields to expand your options or renamed fields to improve clarity — it may even be the case that the new system simply stores certain things differently without giving you any options (rare, but it can happen).
To combat this, a good data migration is preceded by painstaking field mapping, ensuring that all the data will end up in the right places when the migration takes place. When that mapping isn’t done (or just isn’t completed), certain pieces of data can end up in the wrong places, making the situation very confusing. This can also happen when you don’t set up comprehensive rules for record delineation (an entry with a comma can be parsed as two distinct entries, for instance).
If that happens to you, and you have a lot of data in your database, then you’ll likely find yourself putting a lot of time into periodically mapping and updating fields. You’ll need to go into records, determine which piece of data should go where, and move it over. It will get easier the more you understand what happened during the migration, but if you have enough complex records, you might continue finding errors for years to come.
Not Being Able to Trust your All-Important Data
Do you currently have a database? If you do, then I’d wager you’ve kept it tightly controlled since creating it, ensuring that it’s maintained and updated on a near-constant basis to grant the data an outstanding level of integrity. You know that you can trust the accuracy of any given record or field because the system was built the right way and has been live the entire time, with every change of note making its way there.
But once you’ve gone through a bad data migration, all that confidence is gone. Even if everything superficially appears correct, you can’t be sure that any given record is accurate. In the short term, you can refer back to your old system, but what are you going to do: keep it running in perpetuity? And what happens when you start updating the new system but not the old one? You’ll have one outdated system and one potentially-incorrect system.
And while there are various ways to deal with this lack of trust, none is ideal. You can simply proceed with caution, placing less faith in your data, but that will lower its value. You can attempt a corrective data migration, but that will take time and may well cause issues with the data you’ve collected since the first one. You can scour it and update it manually, but that will take a huge amount of time and be very frustrating. You must get it right the first time.
Having to Confirm Things you Should Know
As awkward as it is to address potentially-inaccurate data within your business, it’s vastly more awkward to address it when dealing with clients. Just imagine the troublesome scenario of needing to reach out to a client to ask them to confirm something they told you months ago.
Imagine the impact in the eCommerce world of running a store, building up a set of loyal customers, then buying a new store, emailing all those customers to let them know, and saying what amounts to “Oh, by the way… have you purchased from us before? What’s your address? What topics are you interested in?”.
How will it look to them? At the very least, they’ll think you haven’t really been paying attention — and beyond that, they’ll suspect that their data isn’t safe with you. That isn’t something you can afford to happen. They may even refuse to comply, demanding that you recover the data if you’re so eager to have it.
Increased Stress And Lowered Morale
Everything we’ve looked at so far will inevitably tax the staff members who need to deal with the compromised data. All these new tasks will be added to their usual workloads, and people will start looking to assign blame for the situation. If someone takes responsibility for the bad data migration and resigns, the team can move on, but in a weakened state. If no one takes responsibility then you might end up with a lot of harbored resentment.
Consider that a team that badly botched a data migration is unlikely to have the expertise to suitably handle the aftermath, and you have a scenario in which things are just going to get worse over time, potentially to the point of the entire system needing to be redone and the database painstakingly being reconstructed. Falling profits and more resignations are likely.
As we’ve seen, a bad data migration is likely to have various negative effects in the long term, ranging from minor inconveniences to major obstacles. If you need to migrate your data to a new system and you’re not 100% confident about handling it in-house, be sure to bring in a qualified consultant to handle it for you and make sure you avoid disaster.
