[APM] Throughput charts broken when a filter is set

[MartinKolarik]

Kibana version

7.15.1

APM Server version (if applicable)

7.15.1

Elasticsearch version (if applicable)

7.15.1

Steps to Reproduce

1. Set any kind of filter on the transactions tab.
2. Check that the throughput chart matches the expectations.

Expected Behavior

The chart shows the throughput of the transactions that match the filter.

Actual Behavior

I'm not entirely sure what the charts show but it seems both the big throughput chart and the small one are wrong. In the first screenshot, the big one shows about 10 TPM, the small one shows 1762 TPM for the exact same thing, while the correct value is likely about 180 TPM (based on our regular requests, and based on what I see when the filter is removed, see the second screenshot).

Same time period, no filter. This one shows the correct value:

[elasticmachine]

Pinging @elastic/apm-ui (Team:apm)

[sorenlouv]

Thanks for opening this. I've been able to reproduce, and there is indeed a bug. It looks like transaction metrics are used instead of transactions when the search bar is used. The bug is located here:

kibana/x-pack/plugins/apm/server/lib/helpers/transactions/index.ts

Lines 53 to 78 in 1efcfb6

	export async function getSearchAggregatedTransactions({
	config,
	start,
	end,
	apmEventClient,
	kuery,
	}: {
	config: APMConfig;
	start?: number;
	end?: number;
	apmEventClient: APMEventClient;
	kuery: string;
	}): Promise<boolean> {
	const searchAggregatedTransactions = config.searchAggregatedTransactions;
	if (
	kuery ||
	searchAggregatedTransactions === SearchAggregatedTransactionSetting.auto
	) {
	return getHasAggregatedTransactions({ start, end, apmEventClient, kuery });
	}
	return (
	searchAggregatedTransactions === SearchAggregatedTransactionSetting.always
	);
	}