Fix instant-DDL deadlock on GhostTableMigrated changelog signal

[peterbollen]

Related issue: #1735

Description

Fixes a deadlock that hangs gh-ost during cleanup when --attempt-instant-ddl is used and the instant ALTER succeeds.

initiateApplier writes a GhostTableMigrated changelog row. The streamer's changelog listener callback (onChangelogStateEvent) publishes that signal synchronously via base.SendWithContext(ctx, ghostTableMigrated, true) while holding EventsStreamer.listenersMutex (the send runs inside notifyListeners, which holds the mutex for the whole callback).

On the normal path there is a matching <-mgtr.ghostTableMigrated receiver. The instant-DDL success path, however, returns early right after finalCleanup() and never receives, so the send blocks forever holding listenersMutex. finalCleanup() then closes the binlog reader, whose rows-event decode callback (shouldDecodeRowsEvent) needs the same mutex, and BinlogSyncer.Close() waits for that goroutine to exit → permanent deadlock.

Fix: drain the GhostTableMigrated signal on the instant-DDL success path before finalCleanup(), mirroring the receive on the normal path. The drain is extracted into Migrator.drainGhostTableMigrated() and guarded by !Resume, since resume migrations never emit the signal (initiateApplier only writes it when !Revert && !Resume).

Tests

• TestMigratorDrainGhostTableMigrated — the drain consumes the signal and unblocks the publisher; it is skipped for resume migrations.
• TestEventsStreamerInstantDDLDeadlockIsResolvedByDraining — reproduces the exact deadlock (listener callback blocked on the send while holding listenersMutex; shouldDecodeRowsEvent blocked on the same mutex) and proves that draining the signal resolves it.

Both pass with -race.

In case this PR introduced Go code changes:

• contributed code is using same conventions as original code
• script/cibuild returns with no formatting errors, build errors or unit test errors.

[ggilder]

Independent confirmation of the diagnosis in #1735, plus one issue with the fix as written.

Confirming the bug. We hit this in CI while bumping gh-ost from a 2026-05-14 build to v1.1.11. Our instant-DDL integration tests hung indefinitely — the migration stayed in running forever, never reaching a terminal state, exactly as described here. Worth noting for anyone triaging severity: the deadlock isn't reachable before #1699. shouldDecodeRowsEvent is what introduces a second contender for listenersMutex, so prior to that commit the blocked send merely leaked a goroutine. #1699 is what turns it into a hard hang, which makes this a regression for anyone on --attempt-instant-ddl upgrading across it.

The concern with the fix. drainGhostTableMigrated() uses a bare blocking receive:

func (mgtr *Migrator) drainGhostTableMigrated() {
	if mgtr.migrationContext.Resume {
		return
	}
	<-mgtr.ghostTableMigrated
}

The sender publishes via base.SendWithContext, which abandons the send when the context is cancelled. That means a bare receive on this channel becomes permanently unsatisfiable at exactly the moment the context is cancelled — so this trades the instant-DDL deadlock for an abort-path deadlock on the same channel.

Concretely: AttemptInstantDDL() retries up to --default-retries (#1667), and the heartbeat escalates to PanicAbort after its own successive-failure threshold. If the primary becomes unreachable during those retries, abort() cancels the context, the pending changelog send gives up, and drainGhostTableMigrated() then blocks forever — after the instant DDL may already have been applied. Migrate() never returns, its deferred teardown() never runs, and because the status/throttler tickers exit on finishedMigrating rather than on the context, the process stays alive logging a frozen status line.

Suggestion — mirror #1677 here:

func (mgtr *Migrator) drainGhostTableMigrated() error {
	if mgtr.migrationContext.Resume {
		return nil
	}
	select {
	case <-mgtr.ghostTableMigrated:
		return nil
	case <-mgtr.migrationContext.GetContext().Done():
		return mgtr.checkAbort()
	}
}

For what it's worth, I've opened #1758 applying the same treatment to the other remaining bare receive on this channel — the <-mgtr.ghostTableMigrated on the normal path in Migrate(). The two changes are complementary and touch adjacent lines, so they'll likely need light coordination on whichever lands second. Happy to fold this drain into #1758 instead if that's easier for review, or to help rebase this one (it's currently showing as conflicting).