Revert "Disable downsampling for all aggregated namespaces if any nam…

…espace has downsample option is set to "all : false" (#2)" (#3)

This reverts commit 010a3e1.

.buildkite/docs-docker-compose.yml

@@ -3,7 +3,7 @@ services:
   docs:
     build:
         context: ./
-        dockerfile: ../docker/docs/Dockerfile
+        dockerfile: ../docs/Dockerfile
     volumes:
       - ../:/go/src/github.com/m3db/m3
       - $SSH_AUTH_SOCK:/ssh-agent

.github/pull_request_template.md

@@ -1,8 +1,8 @@
 <!--  Thanks for sending a pull request!  Here are some tips for you:
-1. If this is your first time, please read our contributor guidelines: https://github.com/m3db/m3/blob/master/CONTRIBUTING.md and developer notes https://github.com/m3db/m3/blob/master/DEVELOPMENT.md
+1. If this is your first time, please read our contributor guidelines: https://github.com/m3db/m3/blob/master/CONTRIBUTING.md and developer notes https://github.com/m3db/m3/blob/master/DEVELOPER.md
 2. Please prefix the name of the pull request with the component you are updating in the format "[component] Change title" (for example "[dbnode] Support out of order writes") and also label this pull request according to what type of issue you are addressing. Furthermore, if this is a WIP or DRAFT PR, please create a draft PR instead: https://github.blog/2019-02-14-introducing-draft-pull-requests/
-3. Ensure you have added or ran the appropriate tests for your PR: read more at https://github.com/m3db/m3/blob/master/DEVELOPMENT.md#testing-changes
-4. Follow the instructions for writing a changelog note: read more at https://github.com/m3db/m3/blob/master/DEVELOPMENT.md#updating-the-changelog
+3. Ensure you have added or ran the appropriate tests for your PR: read more at https://github.com/m3db/m3/blob/master/DEVELOPER.md#testing-changes
+4. Follow the instructions for writing a changelog note: read more at https://github.com/m3db/m3/blob/master/DEVELOPER.md#adding-a-changelog
 -->
 
 **What this PR does / why we need it**:

.golangci.yml

@@ -34,8 +34,7 @@ run:
   # won't be reported. Default value is empty list, but there is
   # no need to include all autogenerated files, we confidently recognize
   # autogenerated files. If it's not please let us know.
-  skip-files:
-    - ".*_gen.go$"
+  skip-files: []
 
   # by default isn't set. If set we pass it to "go list -mod={option}". From "go help modules":
   # If invoked with -mod=readonly, the go command is disallowed from the implicit
@@ -109,12 +108,10 @@ linters-settings:
     locale: US
     ignore-words:
       - someword
-  exhaustive:
-    default-signifies-exhaustive: true
   lll:
     # max line length, lines longer will be reported. Default is 120.
     # '\t' is counted as 1 character by default, and can be changed with the tab-width option
-    line-length: 120
+    line-length: 100
     # tab width in spaces. Default to 1.
     tab-width: 1
   unused:
@@ -175,6 +172,7 @@ linters:
     - goconst
     - gocritic
     - gocyclo
+    - godox
     - goimports
     - golint
     - gosimple
@@ -213,19 +211,10 @@ linters:
     - exhaustivestruct
     # We allow cuddling assignment following conditions because there are valid 
     # logical groupings for this use-case (e.g. when evaluating config values).
-    - wsl 
-    # Wrapcheck can cause errors until all callsites checking explicit error
-    # types like io.EOF are converted to use errors.Is instead. Re-enable this
-    # linter once all error checks are upgraded.
-    - wrapcheck
-    # godox prevents using TODOs or FIXMEs which can be useful for demarkation
-    # of future work.
-    - godox 
+    - wsl
     # New line required before return would require a large fraction of the
     # code base to need updating, it's not worth the perceived benefit.
-    - nlreturn 
-    # Opinionated and sometimes wrong.
-    - paralleltest
+    - nlreturn
   disable-all: false
   presets:
     # bodyclose, errcheck, gosec, govet, scopelint, staticcheck, typecheck
@@ -298,4 +287,4 @@ issues:
   max-same-issues: 0
 
   # Show only new issues created after git revision `REV`
-  new-from-rev: master
+  new-from-rev: 5a3a12bba59e058b7a4cee6ec472c3f5786f741b

CHANGELOG.md

@@ -1,41 +1,11 @@
 # Changelog
 
-# 1.0.0
-
-## Overview
-
-This release makes breaking changes to the APIs and configuration to provide a simpler experience both for setup and operating M3.
-
-- New [M3 website](https://m3db.io/).
-- New [M3 documentation](https://m3db.io/docs).
-- Simple [M3DB configuration](https://github.com/m3db/m3/blob/master/src/dbnode/config/m3dbnode-local-etcd.yml) and [guides](https://m3db.io/docs/quickstart/docker/).
-- M3DB [hard limits](https://m3db.io/docs/operational_guide/resource_limits/) limits for high resiliency under load.
-- Bootstrap rearchitecture, now able to boostrap hundreds of millions of recently written datapoints in minutes for reads on restart.
-- Continued focus on baseline performance release-over-release.
-
-## Features
-- **M3DB**: Namespace resolution and retention now configured dynamically via API and stored in etcd instead of being defined statically in M3Coordinator configuration.
-```
-message DatabaseCreateRequest {
-  // ...
-
-  // Optional aggregated namespace to create in 
-  // addition to unaggregated namespace
-  AggregatedNamespace aggregated_namespace = 8;
-}
-```
-- **M3DB**: Minimal configuration file with default settings looks like:
-```
-coordinator: {}
-db: {}
-```
-and includes common settings such as global query limits.
+# 1.0.0 (PROVISIONAL - STILL WORK IN PROGRESS)
 
 ## Backwards Incompatible Changes
 
 ### Configuration
 - **M3DB**: `db.bootstrap.bootstrappers` removed
-- **M3DB**: `db.config` nested under `db.discovery.config` (`discovery` can optionally accept different `type`s of defaults instead of a custom `config`)
 - **M3Coordinator**: `cluster.namespaces.storageMetricsType` removed
 - **M3Coordinator**: `tagOptions.tagOptions` no longer supports `legacy` type
 - **M3Query**: `limits.perQuery.maxComputedDatapoints` removed
@@ -61,15 +31,8 @@ listenAddress: "..."
 - **M3Coordinator**: Removed deprecated URL `/api/v1/placement` in favor of stable preferred URL `/api/v1/services/m3db/placement`
 - **M3Coordinator**: Removed deprecated URL `/api/v1/placement/init` in favor of stable preferred URL `/api/v1/services/m3db/placement/init`
 
-### Package
-- `github.com/m3db/m3/src/x/close` removed in favor of `github.com/m3db/m3/src/x/resource`
-- `github.com/m3db/m3/src/dbnode/clock` removed in favor of `github.com/m3db/m3/src/x/clock`
-- `github.com/m3db/m3/src/x/dice/dice.go` moved to `github.com/m3db/m3/src/dbnode/storage/dice.go`
-- `github.com/m3db/m3/src/x/lockfile/lockfile.go` moved to `github.com/m3db/m3/src/dbnode/server/lockfile.go`
-
 ### Misc
 - **M3Query**: Concept of data point limit enforcers removed in favor of the other remaining query limits (e.g. max series). This also removed metrics `cost_reporter_datapoints`, `cost_reporter_datapoints_counter`, and `cost_reporter_over_datapoints_limit`.
-- Linter enabled
 
 # 0.15.17
 

DEVELOPER.md

@@ -0,0 +1,148 @@
+# Developer Notes
+
+## Setup your development environment
+
+First fork https://github.com/m3db/m3 into your own workspace on Github.
+
+Then create your clone:
+
+```bash
+export working_dir=$GOPATH/src/github.com/m3db
+mkdir -p $working_dir
+
+# Set this to your Github user
+export user="your github profile name"
+
+# Clone your fork
+cd $working_dir
+git clone [email protected]:$user/m3.git
+# or: https://github.com/$user/m3.git
+
+# Set upstream
+cd m3
+git remote add upstream [email protected]:m3db/m3.git
+# or: https://github.com/m3db/m3.git
+
+# Never push to upstream master
+git remote set-url --push upstream no_push
+
+# Check if it makes sense:
+git remote -v
+```
+
+Install dependencies:
+
+```bash
+cd $working_dir/m3
+
+make install-vendor-m3
+```
+
+If everything is setup correctly you should be able to build `m3dbnode`:
+
+```bash
+make m3dbnode
+```
+
+## Running the M3 stack locally
+
+Follow the instructions in [this README](./scripts/development/m3_stack/README.md).
+
+## Testing Changes
+
+M3 has an extensive, and ever increasing, set of tests to ensure we are able to validate changes. More notes about the various testing strategies we employ can be found in `TESTING.md`. An unfortunate consequence of the number of tests is running the test suite takes too long on a developer's laptop. Here's the workflow most developers employ to be productive. Note: take this as a suggestion of something that works for some people, not as a directive. Do what makes you enjoy the development process most, including disregarding this suggestion!
+
+Once you have identified a change you want to make, and gathered consensus by talking to some devs, go ahead and make a branch with the changes. To test your changes:
+
+(0) If you have updated an interface that has been mocked, you need to update the generated `gomock `files.
+
+```shell
+# Generate mocks for all top level packages
+make mock-gen
+
+# If you just want to generate it for a single package,
+# replace xyz with the package you want to generate files for, e.g. dbnode
+make mock-gen-xyz
+```
+
+(1) Run unit tests locally
+```
+go test ./... -v
+```
+
+This usually runs within 1minute on most laptops. And catches basic compilation errors quickly.
+
+(2) Submit to CI, and continue to develop further.
+
+(3) Once the CI job finishes - investigate failures (if any), and reproduce locally.
+
+For e.g. if a Unit Tests `TestXYZ` in package `github.com/m3db/m3/xyz` failed, run the following:
+
+```
+$ go test ./xyz -run TestXYZ -race -v
+```
+
+Similarly, for 'Big Unit Tests'/'Integration' tests, you may have to provide an additional build tag argument, e.g.
+
+```
+# example integration test
+$ go test ./integration -tags integration -run TestIndexBlockRotation -v
+
+# example big unit test
+$ go test  -tags big ./services/m3dbnode/main -run TestIndexEnabledServer -v
+```
+
+(4) Fix, rinse, and repeat.
+
+(5) Polish up the PR, ensure CI signs off, and coverage increases. Then ping someone to take a look and get feedback!
+
+## Adding a Changelog
+
+When you open a PR, if you have made are making a significant change you must also update the contents of `CHANGELOG.md` with a description of the changes and the PR number.  You will not know the PR number ahead of time so this must be added as a subsequent commit to an open PR.
+
+The format of the change should be:
+```
+- TYPE **COMPONENT:** Description (#PR_NUMBER)
+```
+
+The `TYPE` should be ommitted if it is a feature.  If it is not a feature it should be one of: `FIX`, `PERF`.  New types of changes should be added to this document before used if they cannot be categorized by the existing types.
+
+Here is an example of an additiong to the pending changelog:
+
+```
+# 0.4.5 (unreleased)
+
+- FIX **DB** Index data race in FST Segment reads (#938)
+```
+
+## Building the Docs
+
+The `docs` folder contains our documentation in Markdown files. These Markdown files are built into a static site using
+[`mkdocs`](https://www.mkdocs.org/) with the [`mkdocs-material`](https://squidfunk.github.io/mkdocs-material/) theme.
+Building the docs using our predefined `make` targets requires a working Docker installation:
+
+```
+# generate the docs in the `site/` directory
+make docs-build
+
+# build docs and serve on localhost:8000 (with live reload)
+make docs-serve
+
+# build the docs and auto-push to the `gh-pages` branch
+make docs-deploy
+```
+
+## M3 Website
+
+The [M3 website](https://m3db.io/) is hosted with Netlify. It is configured to run `make site-build` and then serving the contents of the `/m3db.io` directory. The site is built and republished every time
+there is a push to master.
+
+## Cutting a release
+
+1. Ensure you have a Github API access token with the `repo` scope
+2. Checkout the commit you want to release (do not release commits that don't exist on master)
+3. Create a tag with the version you would like to release. Ex: `git tag -a v0.7.0 -m "v0.7.0"`
+4. Push the tag. Ex: `git push origin v0.7.0`
+5. Run `make GITHUB_TOKEN=<GITHUB_API_ACCESS_TOKEN> release`
+6. Update `CHANGELOG.md` and commit it to master
+7. Copy and paste the text from `CHANGELOG.md` into the release notes [on Github](https://github.com/m3db/m3/releases)