Obtain a lock on filePathPrefix on dbnode startup. Close #641

[edkolev]

As discussed in #641, with this PR a lock on filePathPrefix will be acquired when starting a db node. Two dbnode processes will not be able to run with the same filePathPrefix.

In case another process already has a lock, this error message would be printed:

16:07:39.082121[F] could not acquire lock on /tmp/m3db/.lock: failed obtaining lock: resource temporarily unavailable

When the dbnode exits gracefully, the lock file will be removed (using a defer).

When the dbnode doesn't exit gracefully, another dbnode process will be able to obtain the lock. A bit more details: there's a distinction b/w the lock file which resides on the FS, and the lock which resides in memory; when the dbnode is killed ungracefully the lock file will remain on the FS, but the lock will be removed from memory; the next time a dbnode starts, it will be able to acquire a lock despite the fact that the lock file exists.

@richardartoul please let me know if this PR can be improved.

[richardartoul]

Looks really good, just a few minor comments

[edkolev]

Also let me know if I need to squash.

[richardartoul]

LGTM, could you fix the merge conflict? We've almost figured out how to run C.I for P.Rs from external contributors but we're not quite there yet but I can run C.I and make sure it passes

[edkolev]

We've almost figured out how to run C.I for P.Rs from external contributors

Nice, let me know if you need a test subject :)

The conflict is now resolved, commits squashed.

[codecov]

Codecov Report

Merging #1376 into master will decrease coverage by 20.9%.
The diff coverage is 45.4%.

@@            Coverage Diff            @@
##           master   #1376      +/-   ##
=========================================
- Coverage    70.7%   49.7%     -21%     
=========================================
  Files         827     669     -158     
  Lines       71224   59496   -11728     
=========================================
- Hits        50384   29623   -20761     
- Misses      17539   27334    +9795     
+ Partials     3301    2539     -762

Flag	Coverage Δ
#aggregator	65.7% <ø> (-16.6%)	⬇️
#cluster	60.7% <ø> (-25.2%)	⬇️
#collector	81.8% <ø> (+18%)	⬆️
#dbnode	51.9% <45.4%> (-28.8%)	⬇️
#m3em	65.3% <ø> (-7.9%)	⬇️
#m3ninx	62.2% <ø> (-12%)	⬇️
#m3nsch	51.1% <ø> (ø)	⬆️
#metrics	?
#msg	?
#query	30.1% <ø> (-35.1%)	⬇️
#x	64.7% <ø> (-11.4%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 41c5bce...7ba2bcc. Read the comment docs.

[richardartoul]

Green build: https://buildkite.com/uberopensource/m3-monorepo-ci/builds/196

Congrats and thank you :) If you're interested in continuing to work on M3 shoot me a message or an email and we can pick a more challenging next task!