scm: implement gitignore

[skshetry]

[UPDATE]: Went to implement scm._reset() function and use dulwich by caching its IgnoreManager. Regarding pygit2, leaving as-is till the migrations.

---

If we start making a lot of is_ignored checks, dulwich is a lot slower than pygit2. I'm toying with the idea of not walking through .gitignore-d files/directories for repo.stage and repo.graph collection.

On master version, the performance on dvc status is following:

  Time (mean ± σ):      4.240 s ±  0.192 s    [User: 3.535 s, System: 0.474 s]
  Range (min … max):    4.141 s …  4.781 s    10 runs

On a modified version that does not traverse through git-ignored directory, the performance with the dulwich/gitpython and pygit2 respectively are:

Benchmark #1: dvc status with dulwich
  Time (mean ± σ):      2.260 s ±  0.389 s    [User: 1.779 s, System: 0.074 s]
  Range (min … max):    1.837 s …  2.883 s    10 runs

Benchmark #2: dvc status with gitpython
  Time (mean ± σ):      1.740 s ±  0.317 s    [User: 874.2 ms, System: 719.3 ms]
  Range (min … max):    1.485 s …  2.604 s    10 runs

Benchmark #3: dvc status with pygit2
  Time (mean ± σ):     461.9 ms ±   8.8 ms    [User: 337.5 ms, System: 47.3 ms]
  Range (min … max):   449.1 ms … 476.3 ms    10 runs

Note that dulwich's performance can be made comparable to pygit2 if we cache the IgnoreManager, but by doing so, we won't be able to use it easily in the API (we need to reset it somewhere/somehow).
Just to compare, this is the performance with IgnoreManager cached.

Benchmark #1: dvc status with dulwich cached
  Time (mean ± σ):     512.1 ms ±  73.2 ms    [User: 387.6 ms, System: 43.3 ms]
  Range (min … max):   436.2 ms … 704.0 ms    10 runs

There is deviation in dulwich for some reason though, but it comparable with pygit2. Another reason to choose dulwich is that, we can clear the state of the .gitignore, but pygit2 does not provide an API to do it (the way is just clearing the backends I think).

• ❗ I have followed the Contributing to DVC checklist.

• 📖 If this PR requires documentation updates, I have created a separate PR (or issue, at least) in dvc.org and linked it here.

Thank you for the contribution - we'll try to review it as soon as possible. 🙏

[skshetry]

Looks like I have to drop this change and make it start using pygit2 by default.

[skshetry]

Okay, decided to implement _reset anyway. :)

[efiop]

Great stuff, @skshetry , thank you! 🙏

[skshetry]

@efiop, the above benchmarks are with #5244. This PR can be considered as a prerequisite for that issue, if we decide to move forward with it.