Update site for v2.1.1 and v2.2.0-alpha release notes #4051

jseldess · 2018-11-15T20:45:24Z

No description provided.

cockroach-teamcity · 2018-11-15T20:45:30Z

This change is

cockroach-teamcity · 2018-11-15T20:49:01Z

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/3d2238560f667e2d0ba9cdec4a74ea710fad355d/

cockroach-teamcity · 2018-11-16T14:41:35Z

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/2840c1f1352d24dfe8605a9ec652f81f6f7bb268/

cockroach-teamcity · 2018-11-16T14:56:21Z

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/3477488746aeeb48535cc0ca7cd5e8409c0c7c45/

jseldess · 2018-11-18T17:41:24Z

@spencerkimball, can you please draft a release note for cockroachdb/cockroach#30143?

jseldess · 2018-11-18T17:42:10Z

@spencerkimball, actually, I can go with the PR description if you're ok with something like cache zone config values to avoid repetitive deserialization.

cockroach-teamcity · 2018-11-18T17:49:37Z

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/325efb5de44cd7d6a6b1e531cad6db3aedc4db5d/

jseldess · 2018-11-18T18:07:46Z

@benesch, I can't figure out how to get the JS in v2.2/install-cockroachdb.md to redirect properly to v2.2/install-cockroachdb-<os>.html. Can you help?

cockroach-teamcity · 2018-11-18T18:09:50Z

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/429d34c5208f5d97360f174ee932a214d3f60152/

jseldess · 2018-11-18T18:14:31Z

@benesch, nevermind. I just had some dates wrong, I think.

cockroach-teamcity · 2018-11-18T18:17:16Z

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/8e74b3da29f639bf6be7709995a39b3153d9e97c/

lnhsingh

LGTM, once minor edits are made and the broken links caught by TeamCity (v2.2-alpha.20191119.html -> v2.2.0-alpha.20181119.html) are fixed.

lnhsingh · 2018-11-18T19:52:58Z

releases/v2.1.1.md

+
+{% include copy-clipboard.html %}
+~~~shell
+docker pull cockroachdb/cockroach:v2.1.1


lnhsingh · 2018-11-18T20:03:28Z

releases/v2.2.0-alpha.20181119.md

+### SQL language changes
+
+- The [`EXPERIMENTAL_RELOCATE`](../v2.2/experimental-features.html) statement no longer temporarily increases the number of replicas in a range more than one above the range's replication factor, preventing rare edge cases of unavailability. [#29684][#29684] {% comment %}doc{% endcomment %}
+- The output of [`SHOW JOBS`](../v2.2/show-jobs.html) now reports ongoing jobs first in start time order, then completed jobs in finished time order. The `running_status` column becomes NULL when the status cannot be determined. [#30917][#30917] {% comment %}doc{% endcomment %}


NULL --> NULL

lnhsingh · 2018-11-18T20:03:31Z

releases/v2.2.0-alpha.20181119.md

+
+{% include copy-clipboard.html %}
+~~~shell
+docker pull cockroachdb/cockroach:v2.2.0-alpha.20181118


lnhsingh · 2018-11-18T20:03:35Z

releases/v2.1.1.md

+
+- Fixed a small memory leak when running distributed queries. [#31759][#31759]
+- The `confkey` column of `pg_catalog.pg_constraint` no longer includes columns that were not involved in the foreign key reference. [#31895][#31895]
+- The [cost-based optimizer](../v2.1/cost-based-optimizer.html) no longer sometimes chooses the wrong index for a scan because of incorrect selectivity estimation. [#32011][#32011]


Remove "sometimes"

lnhsingh · 2018-11-18T20:03:42Z

releases/v2.2.0-alpha.20181119.md

+
+- Improved the layout of the [**Cluster Overview**](../v2.2/admin-ui-cluster-overview-page.html) page for large clusters with many nodes and ranges. [#31512][#31512] {% comment %}doc{% endcomment %}
+- Added the current node ID to the [**Advanced Debugging**](../v2.2/admin-ui-debug-pages.html) page to help identify the current node when viewing the web UI through a load balancer. [#31835][#31835] {% comment %}doc{% endcomment %}
+- The "Non-Table Cluster Data" section of the [**Databases**](../v2.2/admin-ui-databases-page.html) page now includes all non-table data types. Previously, this section only showed Time Series data. [#31830][#31830] {% comment %}doc{% endcomment %}


"Non-Table Cluster Data" section should be Non-Table Cluster Data section, per our Style Guide

lnhsingh · 2018-11-18T20:05:48Z

releases/v2.2.0-alpha.20181119.md

+- [`EXPLAIN ANALYZE`](../v2.2/explain-analyze.html) plans no longer show the processor with ID 0's stats in the Response box. [#31941][#31941]
+- Fixed rare deadlocks during [`IMPORT`](../v2.2/import.html), [`RESTORE`](../v2.2/restore.html), or [`BACKUP`](../v2.2/backup.html). [#31963][#31963]
+- Fixed a panic caused by incorrectly encoded Azure credentials. [#31984][#31984]
+- The [cost-based optimizer](../v2.2/cost-based-optimizer.html) no longer sometimes chooses the wrong index for a scan because of incorrect selectivity estimation. [#31937][#31937]


Remove "sometimes"

lnhsingh · 2018-11-18T20:05:50Z

releases/v2.2.0-alpha.20181119.md

+
+### Bug fixes
+
+- Hash functions with NULL input now return NULL. [#29822][#29822]


NULL -> NULL

cockroach-teamcity · 2018-11-18T23:53:50Z

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/adca85bf54aaf1f0da5d9ce53769c1529bd323b6/

cockroach-teamcity · 2018-11-19T01:49:02Z

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/ccb6330d5a8e9df08d0a2a47d0bd3ac26ba5f563/

cockroach-teamcity · 2018-11-19T01:53:38Z

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/05ae7ee4986f2d43a276c204b2b4df56f154ca0f/

cockroach-teamcity · 2018-11-19T02:21:39Z

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/d011dbf5e7d0beaa71ca05d456354a4b7db97d4e/

cockroach-teamcity · 2018-11-19T02:34:15Z

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/fd814ae8a23cce1dad689173607d1d255f1c1a1d/

cockroach-teamcity · 2018-11-19T02:45:27Z

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/463037e579ccc349c49753237400dabc1bb339f6/

cockroach-teamcity · 2018-11-19T02:57:09Z

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/b21977104d80ed7eb3c9d4d969f79430559eb7c5/

cockroach-teamcity · 2018-11-19T14:57:10Z

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/ac94ddb7913372abe3d67097971ee7d2e1d8d484/

cockroach-teamcity · 2018-11-19T20:50:37Z

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/9da2f0a3dc7bf46563b17cfaee60d8c500b39fe9/

31997: storage/spanlatch: create spanlatch.Manager using immutable btrees r=nvanbenschoten a=nvanbenschoten Informs #4768. Informs #31904. This change was inspired by #31904 and is a progression of the thinking started in #4768 (comment). The change introduces `spanlatch.Manager`, which will replace the `CommandQueue` **in a future PR**. The new type isn't hooked up yet because doing so will require a lot of plumbing changes in that storage package that are best kept in a separate PR. The structure uses a new strategy that reduces lock contention, simplifies the code, avoids allocations, and makes #31904 easier to implement. The primary objective, reducing lock contention, is addressed by minimizing the amount of work we perform under the exclusive "sequencing" mutex while locking the structure. This is made possible by employing a copy-on-write strategy. Before this change, commands would lock the queue, create a large slice of prerequisites, insert into the queue and unlock. After the change, commands lock the manager, grab an immutable snapshot of the manager's trees in O(1) time, insert into the manager, and unlock. They can then iterate over the immutable tree snapshot outside of the lock. Effectively, this means that the work performed under lock is linear with respect to the number of spans that a command declares but NO LONGER linear with respect to the number of other commands that it will wait on. This is important because `Replica.beginCmds` repeatedly comes up as the largest source of mutex contention in our system, especially on hot ranges. The use of immutable snapshots also simplifies the code significantly. We're no longer copying our prereqs into a slice so we no longer need to carefully determine which transitive dependencies we do or don't need to wait on explicitly. This also makes lock cancellation trivial because we no longer explicitly hold on to our prereqs at all. Instead, we simply iterate through the snapshot outside of the lock. While rewriting the structure, I also spent some time optimizing its allocations. Under normal operation, acquiring a latch now incurs only a single allocation - that being for the `spanlatch.Guard`. All other allocations are avoided through object pooling where appropriate. The overhead of using a copy-on-write technique is almost entirely avoided by atomically reference counting immutable btree nodes, which allows us to release them back into the btree node pools when they're no longer needed. This means that we don't expect any allocations when inserting into the internal trees, even with the copy-on-write policy. Finally, this will make the approach taken in #31904 much more natural. Instead of tracking dependents and prerequisites for speculative reads and then iterating through them to find overlaps after, we can use the immutable snapshots directly! We can grab a snapshot and sequence ourselves as usual, but avoid waiting for prereqs. We then execute optimistically before finally checking whether we overlapped any of our prereqs. The great thing about this is that we already have the prereqs in an interval tree structure, so we get an efficient validation check for free. ### Naming changes | Before | After | |----------------------------|-----------------------------------| | `CommandQueue` | `spanlatch.Manager` | | "enter the command queue" | "acquire span latches" | | "exit the command queue" | "release span latches" | | "wait for prereq commands" | "wait for latches to be released" | The use of the word "latch" is based on the definition of latches presented by Goetz Graefe in https://15721.courses.cs.cmu.edu/spring2016/papers/a16-graefe.pdf (see https://i.stack.imgur.com/fSRzd.png). An important reason for avoiding the word "lock" here is that it is critical for understanding that we don't confuse the operational locking performed by the CommandQueue/spanlatch.Manager with the transaction-scoped locking enforced by intents and our transactional concurrency control model. ### Microbenchmarks NOTE: these are single-threaded benchmarks that don't benefit at all from the concurrency improvements enabled by this new structure. ``` name old time/op new time/op delta ReadOnlyMix/size=1-4 706ns ±20% 404ns ±10% -42.81% (p=0.008 n=5+5) ReadOnlyMix/size=4-4 649ns ±23% 382ns ± 5% -41.13% (p=0.008 n=5+5) ReadOnlyMix/size=16-4 611ns ±16% 367ns ± 5% -39.83% (p=0.008 n=5+5) ReadOnlyMix/size=64-4 692ns ±14% 370ns ± 1% -46.49% (p=0.016 n=5+4) ReadOnlyMix/size=128-4 637ns ±22% 398ns ±14% -37.48% (p=0.008 n=5+5) ReadOnlyMix/size=256-4 676ns ±15% 385ns ± 4% -43.01% (p=0.008 n=5+5) ReadWriteMix/readsPerWrite=0-4 12.2µs ± 4% 0.6µs ±17% -94.85% (p=0.008 n=5+5) ReadWriteMix/readsPerWrite=1-4 7.88µs ± 2% 0.55µs ± 7% -92.99% (p=0.008 n=5+5) ReadWriteMix/readsPerWrite=4-4 4.19µs ± 3% 0.58µs ± 5% -86.26% (p=0.008 n=5+5) ReadWriteMix/readsPerWrite=16-4 2.09µs ± 6% 0.54µs ±13% -74.13% (p=0.008 n=5+5) ReadWriteMix/readsPerWrite=64-4 875ns ±17% 423ns ±29% -51.64% (p=0.008 n=5+5) ReadWriteMix/readsPerWrite=128-4 655ns ± 6% 362ns ±16% -44.71% (p=0.008 n=5+5) ReadWriteMix/readsPerWrite=256-4 549ns ±16% 314ns ±13% -42.73% (p=0.008 n=5+5) name old alloc/op new alloc/op delta ReadOnlyMix/size=1-4 223B ± 0% 160B ± 0% -28.25% (p=0.079 n=4+5) ReadOnlyMix/size=4-4 223B ± 0% 160B ± 0% -28.25% (p=0.008 n=5+5) ReadOnlyMix/size=16-4 223B ± 0% 160B ± 0% -28.25% (p=0.008 n=5+5) ReadOnlyMix/size=64-4 223B ± 0% 160B ± 0% -28.25% (p=0.008 n=5+5) ReadOnlyMix/size=128-4 217B ± 4% 160B ± 0% -26.27% (p=0.008 n=5+5) ReadOnlyMix/size=256-4 223B ± 0% 160B ± 0% -28.25% (p=0.079 n=4+5) ReadWriteMix/readsPerWrite=0-4 1.25kB ± 0% 0.16kB ± 0% -87.15% (p=0.008 n=5+5) ReadWriteMix/readsPerWrite=1-4 1.00kB ± 0% 0.16kB ± 0% -84.00% (p=0.079 n=4+5) ReadWriteMix/readsPerWrite=4-4 708B ± 0% 160B ± 0% -77.40% (p=0.079 n=4+5) ReadWriteMix/readsPerWrite=16-4 513B ± 0% 160B ± 0% -68.81% (p=0.008 n=5+5) ReadWriteMix/readsPerWrite=64-4 264B ± 0% 160B ± 0% -39.39% (p=0.008 n=5+5) ReadWriteMix/readsPerWrite=128-4 221B ± 0% 160B ± 0% -27.60% (p=0.079 n=4+5) ReadWriteMix/readsPerWrite=256-4 198B ± 0% 160B ± 0% -19.35% (p=0.008 n=5+5) name old allocs/op new allocs/op delta ReadOnlyMix/size=1-4 1.00 ± 0% 1.00 ± 0% ~ (all equal) ReadOnlyMix/size=4-4 1.00 ± 0% 1.00 ± 0% ~ (all equal) ReadOnlyMix/size=16-4 1.00 ± 0% 1.00 ± 0% ~ (all equal) ReadOnlyMix/size=64-4 1.00 ± 0% 1.00 ± 0% ~ (all equal) ReadOnlyMix/size=128-4 1.00 ± 0% 1.00 ± 0% ~ (all equal) ReadOnlyMix/size=256-4 1.00 ± 0% 1.00 ± 0% ~ (all equal) ReadWriteMix/readsPerWrite=0-4 38.0 ± 0% 1.0 ± 0% -97.37% (p=0.008 n=5+5) ReadWriteMix/readsPerWrite=1-4 24.0 ± 0% 1.0 ± 0% -95.83% (p=0.008 n=5+5) ReadWriteMix/readsPerWrite=4-4 12.0 ± 0% 1.0 ± 0% -91.67% (p=0.008 n=5+5) ReadWriteMix/readsPerWrite=16-4 5.00 ± 0% 1.00 ± 0% -80.00% (p=0.008 n=5+5) ReadWriteMix/readsPerWrite=64-4 2.00 ± 0% 1.00 ± 0% -50.00% (p=0.008 n=5+5) ReadWriteMix/readsPerWrite=128-4 1.00 ± 0% 1.00 ± 0% ~ (all equal) ReadWriteMix/readsPerWrite=256-4 1.00 ± 0% 1.00 ± 0% ~ (all equal) ``` There are a few interesting things to point about about these benchmark results: - The `ReadOnlyMix` results demonstrate a fixed improvement, regardless of size. This is due to the replacement of the hash-map with a linked-list for the readSet structure. - The `ReadWriteMix` is more interesting. We see that the spanlatch implementation is faster across the board. This is especially true with a high write/read ratio. - We see that the allocated memory stays constant regardless of the write/read ratio in the spanlatch implementation. This is due to the memory recylcing that it performs on btree nodes. It is not the case for the CommandQueue implementation. Release note: None 32416: scripts: enhance the release notes r=knz a=knz Fixes #25180. With this the amount of release notes for the first 2.2 alpha in cockroachdb/docs#4051 is reduced to just under two pages. Also this PR makes it easier to monitor progress during the execution of the script. Co-authored-by: Nathan VanBenschoten <[email protected]> Co-authored-by: Raphael 'kena' Poss <[email protected]>

jseldess added the in progress label Nov 15, 2018

knz mentioned this pull request Nov 16, 2018

scripts: enhance the release notes cockroachdb/cockroach#32416

Merged

jseldess force-pushed the release-notes branch from 3d22385 to 2840c1f Compare November 16, 2018 14:38

jseldess force-pushed the release-notes branch from 2840c1f to 3477488 Compare November 16, 2018 14:53

jseldess force-pushed the release-notes branch from 3477488 to 325efb5 Compare November 18, 2018 17:46

jseldess changed the title ~~[WIP] Update site for v2.1.1 and v2.2.0-alpha release notes~~ Update site for v2.1.1 and v2.2.0-alpha release notes Nov 18, 2018

jseldess force-pushed the release-notes branch from 429d34c to 8e74b3d Compare November 18, 2018 18:13

lnhsingh reviewed Nov 18, 2018

View reviewed changes

jseldess force-pushed the release-notes branch from 8e74b3d to adca85b Compare November 18, 2018 23:50

jseldess force-pushed the release-notes branch from adca85b to ccb6330 Compare November 19, 2018 01:45

jseldess force-pushed the release-notes branch from ccb6330 to 05ae7ee Compare November 19, 2018 01:50

jseldess force-pushed the release-notes branch from 05ae7ee to d011dbf Compare November 19, 2018 02:15

jseldess force-pushed the release-notes branch from fd814ae to 463037e Compare November 19, 2018 02:42

jseldess force-pushed the release-notes branch from 463037e to b219771 Compare November 19, 2018 02:53

jseldess mentioned this pull request Nov 19, 2018

Improve docs around automatic retries #4044

Merged

jseldess added 4 commits November 19, 2018 15:30

Update site for v2.1.1 and v2.2.0-alpha release notes

543d984

Add new milestone IDs to script for creating issues from release notes

c69b476

Change mentions of v2.1 to v2.2

e19f909

Port #3958 to v2.2 docs

9da2f0a

jseldess force-pushed the release-notes branch from ac94ddb to 9da2f0a Compare November 19, 2018 20:46

jseldess merged commit e445b6d into master Nov 19, 2018

jseldess deleted the release-notes branch November 19, 2018 20:52

jseldess removed the in progress label Nov 19, 2018


		### Bug fixes

		- Hash functions with NULL input now return NULL. [#29822][#29822]

Update site for v2.1.1 and v2.2.0-alpha release notes #4051

Update site for v2.1.1 and v2.2.0-alpha release notes #4051

Conversation

jseldess commented Nov 15, 2018

cockroach-teamcity commented Nov 15, 2018

cockroach-teamcity commented Nov 15, 2018

cockroach-teamcity commented Nov 16, 2018

cockroach-teamcity commented Nov 16, 2018

jseldess commented Nov 18, 2018

jseldess commented Nov 18, 2018

cockroach-teamcity commented Nov 18, 2018

jseldess commented Nov 18, 2018

cockroach-teamcity commented Nov 18, 2018

jseldess commented Nov 18, 2018

cockroach-teamcity commented Nov 18, 2018

lnhsingh left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cockroach-teamcity commented Nov 18, 2018

cockroach-teamcity commented Nov 19, 2018

cockroach-teamcity commented Nov 19, 2018

cockroach-teamcity commented Nov 19, 2018

cockroach-teamcity commented Nov 19, 2018

cockroach-teamcity commented Nov 19, 2018

cockroach-teamcity commented Nov 19, 2018

cockroach-teamcity commented Nov 19, 2018

cockroach-teamcity commented Nov 19, 2018