Decol Futures: The Dreaded Data Migration

A newsletter to learn about practical ways to decolonize your research and data work-lives with byte-sized drabbles about the daily life of a data professional.

Some of my clients this month are experiencing forced data migrations and their problem is they don’t know where to start. I get asked things like: Which system is better and how do I get my stuff on it? Or how can my unique data needs fit in a new system when my colleague’s needs are so different?

Data migrations are inevitable. Database systems (like a CMS, DAM, or ILL) are a lot like car manufacturers. All car dealers will proclaim they offer the best prices, car choices, and driving options, yet from a consumer standpoint, a lot of that fluff is just marketing.

Database system vendors - like Gallery Systems, Bynder, Ex Libris - are vendors selling a product. They’re incentivized to make a generalist system to fit many data needs for many types of users (museum vs an art foundation vs a tech startup). And the design choices of who the primary user is matters greatly when you talk about a migration.

The top problem I see with data migrations is the poor usability and user experience design. Literally, staff who stare at the content as their day job, can’t find items or it’s challenging to edit or upload items. Migrations often consider user needs, but fail to consider staff needs. Staff are users too. I argue staff needs should be prioritized in migrations because they mediate and preserve and manage this data. They’re the front lines of your organization.

So what do you do? You’re far beyond preventative measures. Try this:

• Scooby Doo It - look for clues to the database software and hardware you have currently. Who owns your server? Is your system on the cloud? Who has root access? There’s always developer documentation online. Find someone who can read it.

• Assess - What data issues do you have? You will need to map your data fields like title or date to a new system. Usually this means converting date formats (7/28/24 is now written as 240728), and accepting that your records or objects don’t have clear cut thematic metadata fields.

• Write a plan - Decide how you will map data from your current system to the next, who will do it, and what the timeline is. Most importantly, prepare for cleanup after data is moved to its new rental home.

Educational Opportunities

A short list of things to explore!

[Paid Course] Learn the Developer Logic Behind Enterprise Database Migrations

You’ll be more successful if you can speak the same words and know the logic of how a software developer practically migrate data. Most databases are going to be SQL-based and cloud-based, so this course is p e r f e c t. Coursera is relatively cheap (less than $100) and you get a certificate.

[Free Webinar] Learn about migrating to AWS (everyone uses this now for backup storage)

Legit. Know what Amazon Web Services (AWS) is. You don’t need to be a developer to do a AWS data migration. Full stop. It’s cool to know how. It’ll help you in job interviews, and most organizations have some kind of redundancy or backup storage - that could be a service like Iron Mountain or something “in house” like AWS.

[Thinking Fuel] Read up on “Technical Debt,” a concept to mean past tech/system/logic decisions that haunt your future data plans

This is a free to read article published in The American Archivist. Technical debt is a great way to conceptualize data migrations. Every migration has to deal with past cataloging workflows, or the system configurations, or metadata schemes.

Let’s Create!

I’m open to collaborations, freelance gigs, and conversations about the ideas I shared. Feel free to get in touch and comment on the newsletter.