Batch jobs moved student data between systems on a schedule, so the systems around the student record could be behind it. The fix was an integration layer that moves each change when it happens.
Consulting engagementabout 2 min read
The situation
The student information system was the system of record, and the systems around it were fed by batch processes. A change made during the day reached the systems downstream after the next batch had run.
In my experience a delay like that has a cost that does not show up in a diagram. People stop trusting the second screen and start checking both. And when a scheduled job fails, the first sign is often a question from someone who noticed.
What I built
An integration layer that keeps student data in step across systems as changes happen.
Events in place of schedules. A change in the student system raises an event, and the integration layer carries it to the systems that need it.
Serverless on Azure. Functions that run when there is work to do, with no servers to patch.
The supported interfaces only. Reads and writes go through the student system’s own APIs, which is the route the vendor supports through upgrades.
Bulk loads handled separately. Large batches of data, which events are the wrong tool for, move through SSIS, a separate bulk-loading tool.
A change is captured once, queued, and delivered to each system that needs it.
Safeguards
Moving data faster only helps if it is still right. My standard for a layer like this is the same every time: every message is logged, delivery can be retried without creating a duplicate, and anything that cannot be delivered is reported to a person instead of being lost.
The result
Batch processes were replaced with near-live data movement, so the systems around the student record keep up with it during the working day.
What I would tell another school
Do not start with the technology. Start by listing every place a staff member retypes something or waits for tomorrow. Those are your integrations, in priority order.
Printed from https://www.datazak.com/work/real-time-student-data-sync/. An address in parentheses after a link that begins with a slash is a page on https://www.datazak.com.
More work
Full-time architect role, then a consulting engagement
A check-only report is read and approved first; then one command corrects financial aid records for a whole student population, can be run again safely, and leaves before-and-after evidence for every student.
Read the case study
Have a system that needs untangling?
Tell me what you run and where it hurts. The first call is 30 minutes, free, and you will leave with a clear next step whether or not we work together.