nullEDGE – Avoiding data migration pitfalls with Slava Savitsky

The first rule of the Data Migration Club is you do not migrate data.

The second rule of the Data Migration Club is you DO NOT MIGRATE DATA!

Or as Slava quotes:

A flawless data migration is one that never takes place…

Sometimes you can’t avoid it, and that’s when Slava Savitsky’s session at nullEDGE can save you from a lot of pain.

Key takeaways

Understand and challenge the scope

It might start with “We just need to migrate incidents“, but that’s just the tip of the iceberg.

What the stakeholders usually mean is “We need to migrate incidents and…”:

• Work notes, comments and email threads.
• Attachments (often huge).
• SLAs.
• Audit history and activity logs.
• Related tasks and child records.
• Other references and relationships.

Besides, “incidents” might not be just “incidents” in the ServiceNow definition. What people call “incidents” is probably a mix of incidents, requests, changes, security incidents, cases…

That’s why it’s important to map tables before you map fields.

Slava proposes a “simple” workflow:

Start early → Explore → Challenge → Validate

And adds an analogy about moving to a new house. It’s a great opportunity to leave behind the junk.

Do we really care about emails from 10 years ago?

Do we actually still need all those huge attachments?

Don’t accept a vague “we must keep everything for legal reasons” without talking to the actual legal/compliance teams. It’s normal to be afraid of regulations you don’t fully understand. Your job is to manage that and make decisions based on facts, not on fear.

Sometimes, you can keep an old system in read-only mode for a while or export things to a separate archive instead of polluting your beautiful ServiceNow instance.

Technical advice

Leverage Import Sets via REST

Works well for staged loads and direct integration with the source system, and you can manage both records and attachments.

By default, Import Set rows are deleted after seven days by the scheduled job “Import Set Deleter”:

var ic = new ImportSetCleaner();
// delete import sets and their related import set rows
// that were created more than or equal to 7 days ago
ic.setDaysAgo(7);
ic.clean();
Code language: JavaScript (javascript)

Slava suggests temporarily disabling it until sign-off. Alternatively, you can increase the value. The idea is to be able to review import data for troubleshooting.

Analyse → fix → re‑import rather than delete and start again (it’s much faster).

Use stable, unique identifiers from the source and configure coalesce in transform maps so re‑imports update records instead of creating duplicates.

What shocked me is the idea of giving process owners access to the mapping tables so that they can adjust them and understand why specific records ended up where they did.

Attachments

The system property glide.rest.max_content_length manages the payload limits (default: 10MB, min: 1MB, max: 25MB).

Warning: Base64 encoding can increase the size of the attachment by 1/3 over the original size.

Access controls can be tricky as well. In ServiceNow, if you can read a record, you can read all its attachments. You might need to move attachments to child tasks if you want to have more control over visibility.

Numbering

Don’t reuse numbering from external systems and try to avoid collisions.

Choices

Legacy data usually has values that you no longer want, but don’t deactivate the values until you complete the migration.

Keep active and state in sync

And don’t reinvent the wheel, you can manage this with dictionary attributes instead of custom scripts:

• close_states – inactive state integer values – task state field – Used by the TaskStateUtil API to identify the list of inactive state values delimited by semicolons (;).

By the way, have you ever wondered why there are a “state” and an “incident_state” field?

User IDs

Emails are not guaranteed to be unique and can be longer than 40 characters.

And don’t even think about increasing the User ID max length!

Data policies

They exist for a reason, but rules designed for new records may block valid legacy data. You might need to disable or adjust them temporarily.

Final thoughts

There’s also a special ring in hell for people whose migration strategy is “push to production and fix it if users complain later”.

Data corruption is exponentially harder to fix once it spreads.

If something has to go wrong, better in DEV than in TEST, and definitely not in PROD.

References

• Slava Savitsky: LinkedIn | Website | Telegram
• nullEDGE website
• nullEDGE YouTube channel
• My LinkedIn posts #nullEDGEAdvent (any feedback is welcomed!)
• Intro to my nullEDGE advent calendar adventure