kvserver: don't PUSH_ABORT encountered intent txns during GC

[erikgrinaker]

When the GC queue is cleaning up intents, it appears to push their txns with PUSH_ABORT:

cockroach/pkg/kv/kvserver/gc_queue.go

Lines 476 to 477 in 024ad2f

	intentCount, err := repl.store.intentResolver.
	CleanupIntents(ctx, intents, gcTimestamp, roachpb.PUSH_ABORT)

This can be very disruptive, and also block for a significant time. It does have an age threshold:

cockroach/pkg/kv/kvserver/gc/gc.go

Lines 48 to 53 in 89c91e7

	// IntentAgeThreshold is the threshold after which an extant intent
	// will be resolved.
	var IntentAgeThreshold = settings.RegisterDurationSetting(
	"kv.gc.intent_age_threshold",
	"intents older than this threshold will be resolved when encountered by the GC queue",
	2*time.Hour,

But aborting txns that have been doing work for >2h seems particularly disruptive.

I suggest we use PUSH_TOUCH here, to leave active txns alone. That will also kick off txn recovery for STAGING txns if the txn liveness has expired:

cockroach/pkg/kv/kvserver/batcheval/cmd_push_txn.go

Lines 277 to 281 in 759fdac

	if reply.PusheeTxn.Status == roachpb.STAGING && (pusherWins || recoverOnFailedPush) {
	err := roachpb.NewIndeterminateCommitError(reply.PusheeTxn)
	log.VEventf(ctx, 1, "%v", err)
	return result.Result{}, err
	}

We should also consider simply dropping IntentAgeThreshold and rely solely on the intent txn status if we can efficiently distinguish between active and stale intents via other means.