erpification.com  ·  the definition, and the service

Erpification  /  Guides  /  The cutover weekend runbook

The cutover weekend runbook

Cutover plans run forwards: freeze, load, reconcile, test, go. This one is built backwards, from the last hour at which you can still change your mind. If your plan does not name that hour, it does not have a rollback, whatever its rollback section says.

Rollback is modelled as a database restore. That is the wrong model. You can restore the legacy database. You cannot restore the payment file the bank has accepted, the invoice in your customer's payables queue, or the parcel on the carrier's truck. The real point of no return is the first document the new system sends outside the building. Build the weekend around that hour.

The irreversibility ledger

Build this table in the week before the window, one row per outbound emission the new system can produce, with the planned time of each. The earliest planned time is your rollback deadline. Every emission is switched off when the window opens and switched back on individually, by name, after its own gate. That is also how the deadline moves: by delaying an emission.

EmissionHow it is reversedWho must be called, and how long their window stays open
Bank payment file transmittedA recall request, item by item, on a file that may already be in the clearing cycleYour bank's payments team, by telephone. Closes at the cut-off stated in your service agreement, usually the same business day
EDI 810 invoice sentA cancellation or replacement transaction the partner's system must acceptThe customer's EDI coordinator, not their buyer. Many systems refuse a replacement once the invoice is matched to a receipt
EDI 856 shipping notice sentA cancel notice, then a resend, before the goods arriveThe customer's receiving dock. Closes when the truck arrives
Carrier manifest closedVoid each label, which the carrier may still billYour carrier account representative. Usually until the end of the carrier's billing day, and voiding a label never retrieves a parcel already collected
Invoice emailed with a new-system numberA credit note in the new system, then a re-issue from legacy under a different numberYour controller and the customer's payables contact. Never fully closes: their ledger now holds a document number you cannot withdraw
Card authorisation captured on the new gatewayVoid before settlement, refund afterwards, with fees that may not come backThe gateway support line and your merchant acquirer. Voids end at settlement; after that the cardholder sees a refund
E-invoice or tax-portal submission acceptedA cancellation filing under the authority's own rulesYour tax adviser. The authority sets the window, sometimes days, sometimes one
Serial or lot number consumedManual reversal of the consumption, plus physical re-labelling if the number was printedThe warehouse lead for that site. The serial leaves stock the moment the goods ship
Stock picked against a new-system pick listA physical put-away and a counted legacy adjustmentThe site inventory controller. Closes when the pick is shipped or short-picked

None of those reversals is yours to perform alone, and several depend on somebody answering a telephone on a Sunday night. That is the real cost of a rollback.

The deadline arithmetic, and the test it has to pass

The deadline is a subtraction, run backwards from the moment the business has to be open.

  1. Reopenthe hour the business must transact
  2. Less commsnotify customers, carriers, the bank
  3. Less re-validationprove legacy is transactable again
  4. Less restorethe duration measured in the rehearsal
  5. Deadlinerollback must be called by this time
  6. Test itcompare with the load finish time

Run it both waysForwards a plan always looks feasible. Backwards it often is not. Use the restore duration timed in the rehearsal on production-scale data, never the vendor’s estimate: a restore slows with volume and the estimate does not.

Now invert it. If the rollback deadline falls before the final load is scheduled to finish, the plan has no rollback. It has a hope. There are two honest responses: lengthen the window, or cut the scope that goes live in it. Patching the estimate is not one of them. Run the test on the plan in front of you. Note first what the test needs in order to be runnable at all: a restore duration measured on production-scale data. A plan that cannot state that number cannot sit the test, which means its rollback is an assertion rather than a plan.

A mock cutover is recommended everywhere. The question worth asking first is what result would force a replan, because a rehearsal with no failing result is theatre. This is the failing result. The rehearsal produces four timings: the delta extract, the load in dependency order, reconciliation to a signed answer, and a full restore of legacy. If they do not fit inside the window with comms time on top, the rehearsal has done its job by failing. The data migration checklist covers the load order and the reconciliations behind those timings.

Asymmetric authority, signed before the window

Naming one person as the owner of the rollback call is not the same as making the call operable at 03:40, when that person does not answer. Two rules close the gap, both signed before the window opens.

The authority is deliberately asymmetric. The cutover manager may call a rollback alone. The cutover manager may never declare go-live alone. Go-live needs three signatures in the room: the sponsor, the controller for balances, and operations for shipping. The risk-reducing call is allowed to sit lower in the hierarchy than the risk-taking one. Check your own governance document for this: where it sets one quorum for both calls, the safe call is the one that cannot be made quickly.

Silence defaults to rollback. Write the ladder with a clock on it: sponsor, then the named alternate, and after a stated number of minutes with no contact the cutover manager rolls back by default. Default-to-proceed is how a business discovers on Monday that it went live on a decision nobody made.

The decision packet

A gate becomes an argument when the sponsor is handed a status report instead of a decision. The fix is one page, the same six lines at every gate, in the same order, rehearsed so the format is familiar before it matters.

The last line is the one a status report never carries, and the one that turns a debate into a decision. A defect argued about with ninety minutes of rollback left is a different object from the same defect with ten.

Three exits, not two

Cutover is written as a binary, go or roll back. There is a third exit, and it is missing from the plan rather than from reality. Because it is unnamed it gets taken badly: informally, by whoever is loudest in the room, with no end date and nothing written down.

A defect is on the table and the gate is now. Which exit?

Exit one  →  roll back

Only before the first irreversible emission

And only for a defect blocking a whole subledger or a whole shipping lane. Called by the cutover manager alone. Past the deadline this exit does not exist, whatever anyone's seniority.

Exit two  →  hold

New system read-only, or open to one department

Legacy stays the record for one named subdomain, and the freeze continues on that subdomain only. Needs a stated maximum duration and one named person who owns closing it.

Exit three  →  go forward

Forward with a named workaround

The defect has a written manual procedure, a named owner and a date. Without all three it is not a workaround, it is an unrecorded hole in the process.

Each exit needs its message to customers, carriers and the bank drafted before the window opens, because nobody writes well at four in the morning. Three short drafts, with the blanks limited to a date and a scope.

The freeze log and the replay rule

“Freeze the legacy system” is written as if the business stops. It does not. A customer calls, a truck has to leave, somebody authorises an urgent payment. No plan says where that is recorded while both systems are closed.

One numbered log, on paper or in one shared sheet, nothing else. Each line carries a timestamp in the canonical timezone, the document type, the originator and the amount, and is keyed into neither system. After the decision, one named person replays the log into whichever system turned out to be canonical, ticking back to a count.

Two rules make it real. The log is numbered, so a missing line is visible rather than merely absent. And if it passes an agreed number of lines, that is itself a no-go signal: the window is longer than the business can hold its breath. An unlogged freeze is how post-go-live duplicates and missing shipments get manufactured, by a mechanism worth stating plainly: the same urgent order keyed twice by two people who each assumed the other had not, and the shipment nobody keyed at all because it was handled on the telephone.

Disjoint document number ranges

Invoices, credit notes, journals and cheque stock in the new system start at a range that cannot overlap legacy. The reason is specific to rollback. Overlapping ranges make a rollback a collision: two documents carrying one number, one of which a customer already holds. Disjoint ranges make it a gap, which an auditor can see.

Two calls belong with this and sit outside anybody's obvious remit. Somebody tells the bank about the new cheque range in advance, or the first cheque is returned. And somebody files the new e-invoicing sequence with the tax authority before the first invoice: a retrospective filing is a correction, not a registration.

What “within tolerance” has to mean

Go or no-go tables say “reconciled within tolerance” and leave it undefined, or demand everything ties to the penny. Both are unworkable. Split by object.

ObjectRule at the gate
General ledger trial balanceZero variance. No exceptions, no sign-off available
Receivables open itemsZero variance, document by document, not in total
Payables open itemsZero variance, document by document
Bank balanceZero variance against the statement
Inventory valuation roundingTolerance permitted: a stated threshold, a named variance account, the controller's signature
Currency translationTolerance permitted on the same three conditions

Then the diagnostic that makes the table usable at two in the morning. Variance concentrated in a few accounts or items is a mapping error. Fix the map, reload that object, stay inside the window. Variance equal to the sum of the documents dated after the extract is a cut-time error. Patching it writes next month's reconciliation problem; the delta extract is re-run instead.

One timezone, and the order sites freeze

Every row carries one canonical timezone, with local times in brackets. Pick it once, state it at the top, and never write a bare time. A handover reading 02:00 means two different hours to two people, and the one who is wrong is asleep.

Then the sequencing rule. Sites freeze in reverse order of reopening, so the first site to reopen is the last to freeze. Otherwise the site with the earliest Monday morning carries the longest freeze, and its freeze log overflows. Reopen order is set by local business hours, not by site size or readiness.

§

What the window owes hypercare

Two things are settled inside the cutover window and then lived with for months, which is why they belong in this runbook rather than in the support plan that follows it.

The first is a production configuration freeze, with exactly one named person who may approve an emergency change in production. It starts at the first posted transaction, not at the end of the weekend. An untracked change made at 05:00 to get one department moving is what makes every subsequent defect report unreproducible, and a war room cannot triage a defect it cannot reproduce.

The second is the list of what was deferred, written while the detail is still in the room: every defect carried through exit three with its owner and date, every emission still switched off and the gate that will switch it on, every line of the freeze log that was replayed by hand. Reconstructed on the Monday it is guesswork. Written during the window it is the opening agenda of hypercare, where it has to be reckoned with rather than quietly carried.

What hypercare covers after that, and the conditions on which it is allowed to end, are a separate question with their own answer. The part that matters here is only this: a hypercare period given a duration and no exit criteria is a calendar entry, not a gate, and the cutover weekend is where that distinction is either written down or lost. Go-live is the midpoint of an implementation, not the end, which is also the argument of the six stages.

Before any of thisA runbook this detailed is worth writing only if the business needs the system. If that is still open, start with the readiness check, twenty scored statements that say when the honest answer is no, or with outgrown QuickBooks, or outgrown your process, which separates a software problem from a process one.

Read next