clear memory allocated for sampled data when constructing Dataset from text file

[xuchuanyin]

Sample data is useless after BinMapper is constructed, but the corresponding memory is still there before feature extraction is finished.

[jameslamb]

excellent, thanks for this!

I support this change.

Since users can configure how much data is sampled during the calculation of bins by setting parameter bin_construct_sample_cnt, it's definitely possible for the sampled data to have a noticable memory footprint.

However, a reviewer more familiar with C++, like @shiyu1994 or @guolinke , should probably review this before it's merged.

LightGBM/src/io/dataset_loader.cpp

Lines 926 to 927 in 9f13a9c

	std::vector<std::string> DatasetLoader::SampleTextDataFromMemory(const std::vector<std::string>& data) {
	int sample_cnt = config_.bin_construct_sample_cnt;

[jameslamb]

@xuchuanyin I hope you don't mind, I've proposed a re-wording of this pull request's title. Since pull request titles become items in this project's release notes, I've proposed a title that I think will be a bit easier to understand for users of LightGBM who aren't familiar with its internals.

[xuchuanyin]

@jameslamb yeah, the optimized title is much better

[jameslamb]

/gha run r-valgrind

Workflow R valgrind tests has been triggered! 🚀
https://github.com/microsoft/LightGBM/actions/runs/1612905185

Status: success ✔️.

[shiyu1994]

Thanks for your contribution!

Sorry for the late response. I was busy with another project in previous days. And now I'm back.

[github-actions]

This pull request has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.