db: don't delete files during ingest application

Previously, ingestions would trigger obsolete file deletions during the ingest
application step. This has the potential to exacerbate the write "hiccup"/stall
induced by sstable ingestion. The sequence numbers assigned to the ingestion
are not published until after ingestion application. This prevents any
concurrent writes that obtained later sequence numbers from publishing their
own sequence numbers.

When Options.Experimental.MinDeletionRate is configured, this step to delete
obsolete files should have minimal impact. However, when
Experimental.MinDeletionRate is not configured, the ingesting goroutine would
synchronously perform file deletions, extending the length of the commit stall.

Now ingestions don't trigger deletion of obsolete files. The only file deletion
that an ingestion operation itself could trigger is deletion of a manifest that
was made obsolete during a manifest rotation while committing the version edit.
This is very rare, and delaying the deletion until a flush or compaction
completes is okay.

ingest.go

@@ -1108,7 +1108,6 @@ func (d *DB) ingestApply(
 	}
 	d.updateReadStateLocked(d.opts.DebugCheck)
 	d.updateTableStatsLocked(ve.NewFiles)
-	d.deleteObsoleteFiles(jobID, false /* waitForOngoing */)
 	// The ingestion may have pushed a level over the threshold for compaction,
 	// so check to see if one is necessary and schedule it.
 	d.maybeScheduleCompaction()

testdata/event_listener

@@ -215,8 +215,6 @@ close: db/marker.manifest.000006.MANIFEST-000016
 remove: db/marker.manifest.000005.MANIFEST-000014
 sync: db
 [JOB 12] MANIFEST created 000016
-remove: db/MANIFEST-000011
-[JOB 12] MANIFEST deleted 000011
 remove: ext/0
 [JOB 12] ingested L0:000015 (826 B)
 
@@ -283,6 +281,8 @@ close: db/000022.sst
 sync: db
 sync: db/MANIFEST-000016
 [JOB 17] flushed 1 memtable to L0 [000022] (770 B), in 1.0s (2.0s total), output rate 770 B/s
+remove: db/MANIFEST-000011
+[JOB 17] MANIFEST deleted 000011
 [JOB 18] flushing 2 ingested tables
 create: db/MANIFEST-000023
 close: db/MANIFEST-000016