You do not need to be technical to tell a careful migration from a hopeful one. You need seven questions, and the patience to wait for a straight answer to each.
I have been on both sides of this. I was a registrar and a financial aid administrator before I built migrations, so I know what it is like to live with the result. These are the questions I would have wanted my school to ask.
1. What happens if we run it twice?
This is the most revealing question you can ask, and it is where the title comes from.
A good answer: “Nothing. The second run finds everything already correct and changes nothing.”
A warning sign: “We only run it once,” or a pause. If a migration cannot be run twice safely, then it has to be perfect the first time, on cutover night, with no way back. There is a small working model of the good answer in the financial aid migration case study, which you can try for yourself.
2. Can you show me what it will change before it changes anything?
A good answer: “Yes. There is a check-only run. It lists every record it would touch and what it would do, and it changes nothing. You approve that list first.”
A check-only run turns a leap of faith into a document your staff can read. Insist on reading it.
3. Will you check every record, or a sample?
A good answer: “Every record. The check is automated, so there is no reason to sample.”
“We checked a sample” means “we do not know about the rest.” Sampling made sense when checking was done by hand. It does not now.
4. Who checks the result, and against what?
A good answer: “A separate check, built for the purpose, compares the result with the source system. It is not the migration grading its own work.”
The thing that did the work should never be the only thing that checks it. That is true of people, of scripts and of AI agents alike.
5. What do you do with a record the rules cannot decide?
A good answer: “We stop and hand it to a person, with the reason. We do not guess.”
Every real data set has records that do not fit: two conflicting values, a missing date, a history nobody can explain. A careful migration sets them aside. A careless one picks an answer, and you find out later which.
6. Who presses the button in production?
A good answer: “A named person with the authority to do it, after a rehearsal on a copy of your data, with your sign-off on the check-only report.”
Be wary of any plan where the first full run is the real one.
7. What will I be able to show an auditor afterward?
A good answer: “A before-and-after record for every item we changed, and the result of the check for every item we did not.”
If the answer is a summary (“it completed successfully”), you will be the one explaining the details later, without the evidence.
The questions on one page
- What happens if we run it twice?
- Can you show me what it will change before it changes anything?
- Will you check every record, or a sample?
- Who checks the result, and against what?
- What do you do with a record the rules cannot decide?
- Who presses the button in production?
- What will I be able to show an auditor afterward?
Print this page and take it to your next migration meeting. If you would like a second opinion on the answers you get, tell me about the project. The first call is free.