Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Close WriteBatch to prevent memory leak. #3397

Merged
merged 1 commit into from
Mar 10, 2019

Conversation

Stumble
Copy link
Contributor

@Stumble Stumble commented Mar 9, 2019

ClevelDB requires closing when WriteBatch is no longer needed, https://godoc.org/github.com/jmhodges/levigo#WriteBatch.Close

@codecov-io
Copy link

codecov-io commented Mar 9, 2019

Codecov Report

Merging #3397 into master will increase coverage by 0.04%.
The diff coverage is 23.07%.

@@            Coverage Diff             @@
##           master    #3397      +/-   ##
==========================================
+ Coverage   63.85%   63.89%   +0.04%     
==========================================
  Files         215      215              
  Lines       17746    17730      -16     
==========================================
- Hits        11331    11329       -2     
+ Misses       5490     5472      -18     
- Partials      925      929       +4
Impacted Files Coverage Δ
libs/db/types.go 100% <ø> (ø) ⬆️
libs/db/remotedb/remotedb.go 40.83% <0%> (-0.7%) ⬇️
libs/db/remotedb/grpcdb/server.go 0% <0%> (ø) ⬆️
libs/db/go_level_db.go 66.46% <0%> (-0.41%) ⬇️
libs/db/debug_db.go 20.31% <0%> (-0.33%) ⬇️
libs/db/mem_batch.go 92.59% <0%> (-7.41%) ⬇️
libs/db/prefix_db.go 55.62% <0%> (-0.71%) ⬇️
lite/dbprovider.go 77.7% <100%> (+0.15%) ⬆️
state/txindex/kv/kv.go 76.32% <100%> (+0.19%) ⬆️
config/toml.go 59.25% <0%> (-6.7%) ⬇️
... and 26 more

@alexanderbez
Copy link
Contributor

Looks pretty straightforward to me 🙂

@melekes melekes changed the base branch from master to develop March 10, 2019 08:39
Copy link
Contributor

@melekes melekes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@melekes melekes merged commit b021f1e into tendermint:develop Mar 10, 2019
melekes added a commit that referenced this pull request Mar 10, 2019
melekes pushed a commit that referenced this pull request Mar 10, 2019
ClevelDB requires closing when WriteBatch is no longer needed, https://godoc.org/github.com/jmhodges/levigo#WriteBatch.Close

Fixes the memory leak in cosmos/cosmos-sdk#3842
melekes added a commit that referenced this pull request Mar 12, 2019
* libs/db: close batch (#3397)

ClevelDB requires closing when WriteBatch is no longer needed, https://godoc.org/github.com/jmhodges/levigo#WriteBatch.Close

Fixes the memory leak in cosmos/cosmos-sdk#3842

* update changelog and bump version to 0.30.2
@liamsi liamsi mentioned this pull request Mar 12, 2019
4 tasks
liamsi added a commit that referenced this pull request Mar 13, 2019
melekes pushed a commit that referenced this pull request Mar 14, 2019
* bump ABCIVersion due to renaming BlockSizeParams -> BlockParams
(#3417 (comment))

* Move changelog on consensus params entry to breaking

* Add @melekes' suggestion for breaking change in pubsub into upgrading.md

* Add changelog entry for #3351

* Add changelog entry for #3358 & #3359

* Add changelog entry for #3397

* remove changelog entry for #3397 (was already released in 0.30.2)

* move 3351 to improvements

* Update changelog comment
pinkiebell added a commit to leapdao/tendermint that referenced this pull request Apr 14, 2019
commit 4695414393a3e0bc3836fb1cece7f1f3768d3311
Merge: 6cc3f4d8 def5c8cf
Author: Ethan Buchman <[email protected]>
Date:   Fri Apr 12 10:56:03 2019 -0400

    Merge pull request #3548 from tendermint/release/v0.31.4

    Release/v0.31.4

commit def5c8cf124ff58cc9c4a61ea017808a0c81c722
Author: Ismail Khoffi <[email protected]>
Date:   Fri Apr 12 16:48:34 2019 +0200

    address review comments: (#3550)

    - mention ADR in release summary
     - remove [p2p] api changes
     - amend v0.31.3 log to contain note about breaking change

commit b6da8880c22202a7061ae2093ef01819438a136e
Author: Ismail Khoffi <[email protected]>
Date:   Fri Apr 12 14:24:51 2019 +0200

    prepare v0.31.4 release:

     - prep changelog
     - add missing changelog entries
     - fix minor glitch in existing changelog (v0.31.2)
     - bump versions

commit a453628c4e43c67c747d5078704479aa73716182
Author: Martin Dyring-Andersen <[email protected]>
Date:   Fri Apr 12 13:25:14 2019 +0200

    Fix a couple of typos (#3547)

    Fix some typos in p2p/transport.go

commit 4e4224213f3eaa89abeac6686efe4ac970ad63d0
Author: Sean Braithwaite <[email protected]>
Date:   Fri Apr 12 12:32:00 2019 +0200

    adr: Peer Behaviour (#3539)

    * [adr] ADR 037: Peer Behaviour inital draft
    * Update docs/architecture/adr-037-peer-behaviour.md

    Co-Authored-By: brapse <[email protected]>

    * Update docs/architecture/adr-037-peer-behaviour.md
    Co-Authored-By: brapse <[email protected]>

    * [docs] adr-037 Better footnote styling
    * [ADR] ADR-037 adjust Footnotes for github markdown
    * [ADR] ADR-037 fix numbered list

commit b5b3b85697a9c29adaab0e1ab5d1380459e2c5eb
Author: Alexander Simmerl <[email protected]>
Date:   Fri Apr 12 12:31:02 2019 +0200

    Bring back NodeInfo NetAddress form the dead (#3545)

    A prior change to address accidental DNS lookups introduced the
    SocketAddr on peer, which was then used to add it to the addressbook.
    Which in turn swallowed the self reported port of the peer, which is
    important on a reconnect. This change revives the NetAddress on NodeInfo
    which the Peer carries, but now returns an error to avoid nil
    dereferencing another issue observed in the past. Additionally we could
    potentially address #3532, yet the original problem statemenf of that
    issue stands.

    As a drive-by optimisation `MarkAsGood` now takes only a `p2p.ID` which
    makes it interface a bit stricter and leaner.

commit 18d2c45c334d9f22621ea0af0686e2e54937fb40
Author: Anton Kaliaev <[email protected]>
Date:   Fri Apr 12 10:46:07 2019 +0200

    rpc: Fix response time grow over time (#3537)

    * rpc: store validator info periodly

    * increase ValidatorSetStoreInterval

    also

    - unexpose it
    - add a comment
    - refactor code
    - add a benchmark, which shows that 100000 results in ~ 100ms to get 100
    validators

    * make the change non-breaking

    * expand comment

    * rename valSetStoreInterval to valSetCheckpointInterval

    * change the panic msg

    * add a test and changelog entry

    * update changelog entry

    * update changelog entry

    * add a link to PR

    * fix test

    * Update CHANGELOG_PENDING.md

    Co-Authored-By: melekes <[email protected]>

    * update comment

    * use MaxInt64 func

commit c3df21fe827aad0447bb59a9546c32db8406a5fd
Author: Anton Kaliaev <[email protected]>
Date:   Thu Apr 11 18:59:14 2019 +0300

    add missing changelog entry (#3544)

    * add missing changelog entry

commit bcec8be035a89b5d08ee00b8cc7115017a58c7e0
Author: Anton Kaliaev <[email protected]>
Date:   Thu Apr 11 15:32:16 2019 +0200

    p2p: do not log err if peer is private (#3474)

    * add actionable advice for ErrAddrBookNonRoutable err

    Should replace https://github.com/tendermint/tendermint/pull/3463

    * reorder checks in addrbook#addAddress so

    ErrAddrBookPrivate is returned first

    and do not log error in DialPeersAsync if the address is private
    because it's not an error

commit 9a415b057238d118c1edf23d673ebf9ce4f8d2ae
Author: Anton Kaliaev <[email protected]>
Date:   Tue Apr 9 18:21:35 2019 +0200

    docs: abci#Commit: better explain the possible deadlock (#3536)

commit 40da355234bda9d510dca370e94b646450673879
Author: Anton Kaliaev <[email protected]>
Date:   Wed Apr 3 14:56:51 2019 +0200

    docs: fix block.Header.Time description (#3529)

    It's not proposer local time anymore, but a weighted median

    Fixes #3514

commit f965a4db15796dfca4e504d93eaa83760ced4af2
Author: Anton Kaliaev <[email protected]>
Date:   Wed Apr 3 11:22:52 2019 +0200

    p2p: seed mode refactoring (#3011)

    ListOfKnownAddresses is removed
    panic if addrbook size is less than zero
    CrawlPeers does not attempt to connect to existing or peers we're currently dialing
    various perf. fixes
    improved tests (though not complete)
    move IsDialingOrExistingAddress check into DialPeerWithAddress (Fixes #2716)

    * addrbook: preallocate memory when saving addrbook to file

    * addrbook: remove oldestFirst struct and check for ID

    * oldestFirst replaced with sort.Slice
    * ID is now mandatory, so no need to check

    * addrbook: remove ListOfKnownAddresses

    GetSelection is used instead in seed mode.

    * addrbook: panic if size is less than 0

    * rewrite addrbook#saveToFile to not use a counter

    * test AttemptDisconnects func

    * move IsDialingOrExistingAddress check into DialPeerWithAddress

    * save and cleanup crawl peer data

    * get rid of DefaultSeedDisconnectWaitPeriod

    * make linter happy

    * fix TestPEXReactorSeedMode

    * fix comment

    * add a changelog entry

    * Apply suggestions from code review

    Co-Authored-By: melekes <[email protected]>

    * rename ErrDialingOrExistingAddress to ErrCurrentlyDialingOrExistingAddress

    * lowercase errors

    * do not persist seed data

    pros:
    - no extra files
    - less IO

    cons:
    - if the node crashes, seed might crawl a peer too soon

    * fixes after Ethan's review

    * add a changelog entry

    * we should only consult Switch about peers

    checking addrbook size does not make sense since only PEX reactor uses
    it for dialing peers!

    https://github.com/tendermint/tendermint/pull/3011#discussion_r270948875

commit 086d6cbe8c4432a1a03d7924ae8352c42dc9cc4f
Merge: e4a03f24 6cc3f4d8
Author: Ethan Buchman <[email protected]>
Date:   Tue Apr 2 16:49:44 2019 -0400

    Merge pull request #3527 from tendermint/v0.31

    Merge V0.31.3 back to develop

commit 6cc3f4d87cae6589aef0ac2b4707f1ebc8a669b2
Merge: 79e9f205 3cfd9757
Author: Ethan Buchman <[email protected]>
Date:   Tue Apr 2 16:45:04 2019 -0400

    Merge pull request #3525 from tendermint/release/v0.31.3

    Release/v0.31.3

commit 3cfd9757a78fc2398c6853aace2541338be8eaef
Author: Ethan Buchman <[email protected]>
Date:   Tue Apr 2 09:14:33 2019 -0400

    changelog and version v0.31.3

commit 882622ec10ba3fdc9f360da65bfb2bbbd13193ed
Author: Ethan Buchman <[email protected]>
Date:   Mon Apr 1 19:59:57 2019 -0400

    Fixes tendermint/tendermint#3522

    * OriginalAddr -> SocketAddr

    OriginalAddr records the originally dialed address for outbound peers,
    rather than the peer's self reported address. For inbound peers, it was
    nil. Here, we rename it to SocketAddr and for inbound peers, set it to
    the RemoteAddr of the connection.

    * use SocketAddr

    Numerous places in the code call peer.NodeInfo().NetAddress().
    However, this call to NetAddress() may perform a DNS lookup if the
    reported NodeInfo.ListenAddr includes a name. Failure of this lookup
    returns a nil address, which can lead to panics in the code.

    Instead, call peer.SocketAddr() to return the static address of the
    connection.

    * remove nodeInfo.NetAddress()

    Expose `transport.NetAddress()`, a static result determined
    when the transport is created. Removing NetAddress() from the nodeInfo
    prevents accidental DNS lookups.

    * fixes from review

    * linter

    * fixes from review

commit 1ecf8148385692da971a46e14091270b23c6dff3
Author: Ethan Buchman <[email protected]>
Date:   Mon Apr 1 19:45:57 2019 -0400

    Fixes tendermint/tendermint#3439

    * make sure we create valid private keys:

     - genPrivKey samples and rejects invalid fieldelems (like libsecp256k1)
     - GenPrivKeySecp256k1 uses `(sha(secret) mod (n − 1)) + 1`
     - fix typo, rename test file: s/secpk256k1/secp256k1/

    * Update crypto/secp256k1/secp256k1.go

commit e4a03f249dcb7d2e833fc20add382b2a5e7f167e
Author: Greg Szabo <[email protected]>
Date:   Mon Apr 1 14:18:18 2019 -0400

    Release message changelog link fix (#3519)

commit 56d8aa42b3ca628c6544cab1f902f8289738c97a
Merge: 0ae41cc6 79e9f205
Author: Ethan Buchman <[email protected]>
Date:   Mon Apr 1 14:17:58 2019 -0400

    Merge pull request #3520 from tendermint/v0.31

    Merge v0.31.2 release back to develop

commit 79e9f205782f1adb94b4150561e414280115cf4e
Merge: a0234aff ab24925c
Author: Ismail Khoffi <[email protected]>
Date:   Mon Apr 1 17:58:28 2019 +0200

    Merge pull request #3518 from tendermint/prepare-release-v0.31.2

    Release v0.31.2

commit ab24925c9432e9eff06b34445b1760217436f87d
Author: Ismail Khoffi <[email protected]>
Date:   Mon Apr 1 12:32:37 2019 +0200

    prepare changelog and bump versions to v0.31.2

commit 0ae41cc663014454184c2d31328b53dd45dd34d1
Author: Greg Szabo <[email protected]>
Date:   Mon Apr 1 11:47:00 2019 -0400

    Fix for wrong version tag (#3517)

    * Fix for wrong version tag (tag on the release branch instead of master)

commit 422d04c8bae80e103474d37bd5a5afd0061e8d72
Author: Ethan Buchman <[email protected]>
Date:   Sun Mar 31 07:14:18 2019 -0400

    Bucky/mempool txsmap (#3512)

    * mempool: resCb -> globalCb

    * reqResCb takes an externalCb

    * failing test for #3509

    * txsMap is sync.Map

    * update changelog

commit 2233dd45bd774905b1ba545eeadea5ebe49d1e31
Author: zjubfd <[email protected]>
Date:   Sat Mar 30 01:47:53 2019 +0800

    libs: remove useless code in group (#3504)

    * lib: remove useless code in group
    * update change log
    * Update CHANGELOG_PENDING.md

    Co-Authored-By: guagualvcha <[email protected]>

commit 9199f3f6132ff24500b538068450eae7d11bef64
Author: Greg Szabo <[email protected]>
Date:   Fri Mar 29 07:57:16 2019 -0400

    Release management using CircleCI (#3498)

    * Release management using CircleCI

    * Changelog updated

commit 6c1a4b513747fed5be06edffd16db4a1c8749041
Author: Ismail Khoffi <[email protected]>
Date:   Thu Mar 28 17:39:09 2019 +0100

    blockchain: comment out logger in test code that causes a race condition (#3500)

commit c7bb9984979b72e81a569b4540a2da1e73c1c470
Merge: d586945d 7b72436c
Author: Ethan Buchman <[email protected]>
Date:   Thu Mar 28 08:07:59 2019 -0400

    Merge pull request #3502 from tendermint/bucky/merge-master

    Bucky/merge master

commit 7b72436c751587141d8de605ccbcec3514dbea77
Merge: d586945d a0234aff
Author: Ethan Buchman <[email protected]>
Date:   Thu Mar 28 08:12:05 2019 -0400

    Merge branch 'master' into bucky/merge-master

commit a0234affb6959a0aec285eebf3a3963251d2d186
Merge: 0d985ede 9390a810
Author: Ethan Buchman <[email protected]>
Date:   Thu Mar 28 07:57:42 2019 -0400

    Merge pull request #3489 from tendermint/release/v0.31.1

    Release/v0.31.1

commit 9390a810ebda0b1aebd90f6e8b8688562bdf7958
Author: Ethan Buchman <[email protected]>
Date:   Wed Mar 27 21:03:28 2019 -0400

    minor changelog updates (#3499)

commit a49d80b89c9df3cb2497adb8175915c654479708
Author: Ismail Khoffi <[email protected]>
Date:   Wed Mar 27 19:12:01 2019 +0100

    catch up with develop and rebase on current release to include #3482

commit ccfe75ec4a701ad7dcaf1f26df772b0881ab1184
Author: Sean Braithwaite <[email protected]>
Date:   Wed Mar 27 18:51:57 2019 +0100

    docs: Fix broken links (#3482) (#3488)

    * docs: fix broken links (#3482)

    A bunch of links were broken in the documentation s they included the
    `docs` prefix.

    * Update CHANGELOG_PENDING

    * docs: switch to relative links for github compatitibility (#3482)

commit d586945d6981262759e0e681e7bc92a74637185d
Author: Sean Braithwaite <[email protected]>
Date:   Wed Mar 27 18:51:57 2019 +0100

    docs: Fix broken links (#3482) (#3488)

    * docs: fix broken links (#3482)

    A bunch of links were broken in the documentation s they included the
    `docs` prefix.

    * Update CHANGELOG_PENDING

    * docs: switch to relative links for github compatitibility (#3482)

commit ae88965ff6ddf174607a02eee83cfc5ed11c6cbb
Author: Ismail Khoffi <[email protected]>
Date:   Wed Mar 27 17:46:29 2019 +0100

    changelog: add summary & fix link & add external contributor (#3490)

commit 233813483642c2ac370110dd42c3291556c26039
Author: Ismail Khoffi <[email protected]>
Date:   Wed Mar 27 16:52:19 2019 +0100

    bump versions

commit 1b33a50e6d07e48f2377db64ce944a435f9f2ec4
Merge: 3c7bb6b5 5fa540bd
Author: Ismail Khoffi <[email protected]>
Date:   Wed Mar 27 16:50:59 2019 +0100

    Merge remote-tracking branch 'remotes/origin/develop' into release/v0.31.1

commit 3c7bb6b571ac47e050a47013e22dbdfd68f02745
Author: Ismail Khoffi <[email protected]>
Date:   Wed Mar 27 16:48:00 2019 +0100

    Add some numbers for #2778

commit 5fa540bdc9eac01cc3df6de01119550d1122114b
Author: Anton Kaliaev <[email protected]>
Date:   Wed Mar 27 16:45:34 2019 +0100

    mempool: add a safety check, write tests for mempoolIDs (#3487)

    * mempool: add a safety check, write tests for mempoolIDs

    and document 65536 limit in the mempool reactor spec

    follow-up to https://github.com/tendermint/tendermint/pull/2778

    * rename the test

    * fixes after Ismail's review

commit 52727863e187e85901e7f2e7285920a701d2a8e6
Author: Ismail Khoffi <[email protected]>
Date:   Wed Mar 27 16:07:03 2019 +0100

    add external contributors

commit e3f840e6a6f81406c44e760e578e48fcd3fedf12
Author: Ismail Khoffi <[email protected]>
Date:   Wed Mar 27 16:04:43 2019 +0100

    reset CHANGELOG_PENDING.md

commit ed63e1f378eaf465b714084fa1edd6ac4761782f
Author: Ismail Khoffi <[email protected]>
Date:   Wed Mar 27 16:03:25 2019 +0100

    Add more entries to the Changelog, fix formatting, linkify

commit 55b7118c981e93b920cead4616b91ec58130e316
Author: Ismail Khoffi <[email protected]>
Date:   Wed Mar 27 15:35:32 2019 +0100

    Prep changelog: copy from pending & update version

commit 5a25b75b1d586c6d9c3fdfc940057f7b85089a90
Author: zjubfd <[email protected]>
Date:   Wed Mar 27 00:13:14 2019 +0800

    p2p: refactor GetSelectionWithBias for addressbook (#3475)

    Why submit this pr:

        we have suffered from infinite loop in addrbook bug which takes us a long time to find out why process become a zombie peer. It have been fixed in #3232. But the ADDRS_LOOP is still there, risk of infinite loop is still exist.
        The algorithm that to random pick a bucket is not stable, which means the peer may unluckily always choose the wrong bucket for a long time, the time and cpu cost is meaningless.

    A simple improvement:
    shuffle bucketsNew and bucketsOld, and pick necessary number of address from them. A stable
    algorithm.

commit a4d9539544ba4377f16e797fea01090bc974e1b5
Author: Anton Kaliaev <[email protected]>
Date:   Tue Mar 26 09:44:49 2019 +0100

    rpc/client: include NetworkClient interface into Client interface (#3473)

    I think it's nice when the Client interface has all the methods. If someone does not need a particular method/set of methods, she can use individual interfaces (e.g. NetworkClient, MempoolClient) or write her own interface.

    technically breaking

    Fixes #3458

commit 1bb8e02a962a1ffd7ce4e7355f648b644105f649
Author: HaoyangLiu <[email protected]>
Date:   Tue Mar 26 16:29:06 2019 +0800

    mempool: fix broadcastTxRoutine leak (#3478)

    Refs #3306, irisnet@fdbb676

    I ran an irishub validator. After the validator node ran several days, I dump the whole goroutine stack. I found that there were hundreds of broadcastTxRoutine. However, the connected peer quantity was less than 30. So I belive that there must be broadcastTxRoutine leakage issue.

    According to my analysis, I think the root cause of this issue locate in below code:

    		select {
    		case <-next.NextWaitChan():
    			// see the start of the for loop for nil check
    			next = next.Next()
    		case <-peer.Quit():
    			return
    		case <-memR.Quit():
    			return
    		}

    As we know, if multiple paths are avaliable in the same time, then a random path will be selected. Suppose that next.NextWaitChan() and peer.Quit() are both avaliable, and next.NextWaitChan() is chosen.

                    // send memTx
    		msg := &TxMessage{Tx: memTx.tx}
    		success := peer.Send(MempoolChannel, cdc.MustMarshalBinaryBare(msg))
    		if !success {
    			time.Sleep(peerCatchupSleepIntervalMS * time.Millisecond)
    			continue
    		}

    Then next will be non-empty and the peer send operation won't be success. As a result, this go routine will be track into infinite loop and won't be released.

    My proposal is to check peer.Quit() and memR.Quit() in every loop no matter whether next is nil.

commit 6de7effb05581f9bea2e8af06e4e74a85c34bc5f
Author: Dev Ojha <[email protected]>
Date:   Tue Mar 26 01:27:29 2019 -0700

    mempool no gossip back (#2778)

    Closes #1798

    This is done by making every mempool tx maintain a list of peers who its received the tx from. Instead of using the 20byte peer ID, it instead uses a local map from peerID to uint16 counter, so every peer adds 2 bytes. (Word aligned to probably make it 8 bytes)

    This also required resetting the callback function on every CheckTx. This likely has performance ramifications for instruction caching. The actual setting operation isn't costly with the removal of defers in this PR.

    * Make the mempool not gossip txs back to peers its received it from

    * Fix adversarial memleak

    * Don't break interface

    * Update changelog

    * Forgot to add a mtx

    * forgot a mutex

    * Update mempool/reactor.go

    Co-Authored-By: ValarDragon <[email protected]>

    * Update mempool/mempool.go

    Co-Authored-By: ValarDragon <[email protected]>

    * Use unknown peer ID

    Co-Authored-By: ValarDragon <[email protected]>

    * fix compilation

    * use next wait chan logic when skipping

    * Minor fixes

    * Add TxInfo

    * Add reverse map

    * Make activeID's auto-reserve 0

    * 0 -> UnknownPeerID

    Co-Authored-By: ValarDragon <[email protected]>

    * Switch to making the normal case set a callback on the reqres object

    The recheck case is still done via the global callback, and stats
    are also set via global callback

    * fix merge conflict

    * Addres comments

    * Add cache tests

    * add cache tests

    * minor fixes

    * update metrics in reqResCb and reformat code

    * goimport -w mempool/reactor.go

    * mempool: update memTx senders

    I had to introduce txsMap for quick mempoolTx lookups.

    * change senders type from []uint16 to sync.Map

    Fixes DATA RACE:

    ```
    Read at 0x00c0013fcd3a by goroutine 183:
      github.com/tendermint/tendermint/mempool.(*MempoolReactor).broadcastTxRoutine()
          /go/src/github.com/tendermint/tendermint/mempool/reactor.go:195 +0x3c7

    Previous write at 0x00c0013fcd3a by D[2019-02-27|10:10:49.058] Read PacketMsg                               switch=3 [email protected]
    :46552 conn=MConn{pipe} packet="PacketMsg{30:2B06579D0A143EB78F3D3299DE8213A51D4E11FB05ACE4D6A14F T:1}"
    goroutine 190:
      github.com/tendermint/tendermint/mempool.(*Mempool).CheckTxWithInfo()
          /go/src/github.com/tendermint/tendermint/mempool/mempool.go:387 +0xdc1
      github.com/tendermint/tendermint/mempool.(*MempoolReactor).Receive()
          /go/src/github.com/tendermint/tendermint/mempool/reactor.go:134 +0xb04
      github.com/tendermint/tendermint/p2p.createMConnection.func1()
          /go/src/github.com/tendermint/tendermint/p2p/peer.go:374 +0x25b
      github.com/tendermint/tendermint/p2p/conn.(*MConnection).recvRoutine()
          /go/src/github.com/tendermint/tendermint/p2p/conn/connection.go:599 +0xcce

    Goroutine 183 (running) created at:
    D[2019-02-27|10:10:49.058] Send                                         switch=2 [email protected]:46193 channel=48 conn=MConn{pipe} m
    sgBytes=2B06579D0A146194480ADAE00C2836ED7125FEE65C1D9DD51049
      github.com/tendermint/tendermint/mempool.(*MempoolReactor).AddPeer()
          /go/src/github.com/tendermint/tendermint/mempool/reactor.go:105 +0x1b1
      github.com/tendermint/tendermint/p2p.(*Switch).startInitPeer()
          /go/src/github.com/tendermint/tendermint/p2p/switch.go:683 +0x13b
      github.com/tendermint/tendermint/p2p.(*Switch).addPeer()
          /go/src/github.com/tendermint/tendermint/p2p/switch.go:650 +0x585
      github.com/tendermint/tendermint/p2p.(*Switch).addPeerWithConnection()
          /go/src/github.com/tendermint/tendermint/p2p/test_util.go:145 +0x939
      github.com/tendermint/tendermint/p2p.Connect2Switches.func2()
          /go/src/github.com/tendermint/tendermint/p2p/test_util.go:109 +0x50

    I[2019-02-27|10:10:49.058] Added good transaction                       validator=0 tx=43B4D1F0F03460BD262835C4AA560DB860CFBBE85BD02386D83DAC38C67B3AD7 res="&{CheckTx:gas_w
    anted:1 }" height=0 total=375
    Goroutine 190 (running) created at:
      github.com/tendermint/tendermint/p2p/conn.(*MConnection).OnStart()
          /go/src/github.com/tendermint/tendermint/p2p/conn/connection.go:210 +0x313
      github.com/tendermint/tendermint/libs/common.(*BaseService).Start()
          /go/src/github.com/tendermint/tendermint/libs/common/service.go:139 +0x4df
      github.com/tendermint/tendermint/p2p.(*peer).OnStart()
          /go/src/github.com/tendermint/tendermint/p2p/peer.go:179 +0x56
      github.com/tendermint/tendermint/libs/common.(*BaseService).Start()
          /go/src/github.com/tendermint/tendermint/libs/common/service.go:139 +0x4df
      github.com/tendermint/tendermint/p2p.(*peer).Start()
          <autogenerated>:1 +0x43
      github.com/tendermint/tendermint/p2p.(*Switch).startInitPeer()
    ```

    * explain the choice of a map DS for senders

    * extract ids pool/mapper to a separate struct

    * fix literal copies lock value from senders: sync.Map contains sync.Mutex

    * use sync.Map#LoadOrStore instead of Load

    * fixes after Ismail's review

    * rename resCbNormal to resCbFirstTime

commit 25a3c8b1724c9611d6edc175b1b0d079f5ee28c1
Author: zjubfd <[email protected]>
Date:   Sun Mar 24 01:08:15 2019 +0800

    rpc: support tls rpc (#3469)

    Refs #3419

commit 85be2a554e7e7752bed0b9409ab153bf04e05e7b
Author: Thane Thomson <[email protected]>
Date:   Fri Mar 22 09:16:38 2019 -0400

     tools/tm-signer-harness: update height and round for test harness (#3466)

    In order to re-enable the test harness for the KMS (see
    tendermint/kms#227), we need some marginally more realistic proposals
    and votes. This is because the KMS does some additional sanity checks
    now to ensure the height and round are increasing over time.

commit 1d4afb179b9660bf13705c67b01e20838a4506eb
Author: Anton Kaliaev <[email protected]>
Date:   Thu Mar 21 11:05:39 2019 +0100

    replace PB2TM.ConsensusParams with a call to params#Update (#3448)

    Fixes #3444

commit 660bd4a53e0dd7642a473689c8686c3f83e3a0ca
Author: tracebundy <[email protected]>
Date:   Wed Mar 20 20:30:49 2019 +0800

    fix comment (#3454)

commit 81b9bdf40010c6a4e336133ec53fe6e4e6089911
Author: Ethan Buchman <[email protected]>
Date:   Wed Mar 20 08:29:40 2019 -0400

    comments on validator ordering (#3452)

    * comments on validator ordering

    * NextValidatorsHash

commit 926127c774a2c9110c4284938411818918ffecac
Author: Anton Kaliaev <[email protected]>
Date:   Wed Mar 20 03:59:33 2019 +0300

    blockchain: update the maxHeight when a peer is removed (#3350)

    * blockchain: update the maxHeight when a peer is removed

    Refs #2699

    * add a changelog entry

    * make linter pass

commit 03085c2da23b179c4a51f59a03cb40aa4e85a613
Author: zjubfd <[email protected]>
Date:   Wed Mar 20 08:18:18 2019 +0800

    rpc: client disable compression (#3430)

commit 7af4b5086af9268f7cc8b41f5a174ade675d8ab4
Author: Anton Kaliaev <[email protected]>
Date:   Wed Mar 20 03:10:54 2019 +0300

    Remove RepeatTimer and refactor Switch#Broadcast (#3429)

    * p2p: refactor Switch#Broadcast func

    - call wg.Add only once
    - do not call peers.List twice!
      * bad for perfomance
      * peers list can change in between calls!

    Refs #3306

    * p2p: use time.Ticker instead of RepeatTimer

    no need in RepeatTimer since we don't Reset them

    Refs #3306

    * libs/common: remove RepeatTimer (also TimerMaker and Ticker interface)

    "ancient code that’s caused no end of trouble" Ethan

    I believe there's much simplier way to write a ticker than can be reset
    https://medium.com/@arpith/resetting-a-ticker-in-go-63858a2c17ec

commit 60b2ae5f5a3e16625af1342e012462448d565394
Author: needkane <[email protected]>
Date:   Wed Mar 20 08:00:53 2019 +0800

    crypto: delete unused code (#3426)

commit a6349f50633a45755041b8230a0f7eb990383cfd
Author: Anca Zamfir <[email protected]>
Date:   Wed Mar 20 01:56:13 2019 +0200

    Formalize proposer election algorithm properties  (#3140)

    * Update proposer-selection.md

    * Fixed typos

    * fixed typos

    * Attempt to address some comments

    * Update proposer-selection.md

    * Update proposer-selection.md

    * Update proposer-selection.md

    Added the normalization step.

    * Addressed review comments

    * New example for normalization section

    Added a new example to better show the need for normalization
    Added requirement for changing validator set
    Addressed review comments

    * Fixed problem with R2

    * fixed the math for new validator

    * test

    * more small updates

    * Moved the centering above the round-robin election

    - the centering is now done before the actual round-robin block
    - updated examples
    - cleanup

    * change to reflect new implementation for new validator

commit 22bcfca87a987dc43ad111a226b072ce09bbeb8d
Merge: 4162ebe8 0d985ede
Author: Ethan Buchman <[email protected]>
Date:   Tue Mar 19 19:54:09 2019 -0400

    Merge pull request #3450 from tendermint/master

    Merge master back to develop

commit 0d985ede28bd6937fa9d3613618e42cab6fc871c
Merge: 97681953 1e346978
Author: Ethan Buchman <[email protected]>
Date:   Tue Mar 19 19:53:37 2019 -0400

    Merge pull request #3417 from tendermint/release/v0.31.0

    Release/v0.31.0

commit 1e3469789dce5a034a21b6e48288f1809a102595
Author: Ismail Khoffi <[email protected]>
Date:   Wed Mar 20 00:45:51 2019 +0100

    Ensure WriteTimeout > TimeoutBroadcastTxCommit (#3443)

    * Make sure config.TimeoutBroadcastTxCommit < rpcserver.WriteTimeout()

    * remove redundant comment

    * libs/rpc/http_server: move Read/WriteTimeout into Config

    * increase defaults for read/write timeouts

    Based on this article
    https://www.digitalocean.com/community/tutorials/how-to-optimize-nginx-configuration

    * WriteTimeout should be larger than TimeoutBroadcastTxCommit

    * set a deadline for subscribing to txs

    * extract duration into const

    * add two changelog entries

    * Update CHANGELOG_PENDING.md

    Co-Authored-By: melekes <[email protected]>

    * Update CHANGELOG_PENDING.md

    Co-Authored-By: melekes <[email protected]>

    * 12 -> 10

    * changelog

    * changelog

commit 5f68fbae37e11b01f4d2e5c33b182401a7283e4a
Merge: 551b6322 e276f35f
Author: Ethan Buchman <[email protected]>
Date:   Tue Mar 19 19:25:26 2019 -0400

    Merge pull request #3449 from tendermint/ismail/merge_develop_into_release/0.31.0

    Merge develop into release/0.31.0

commit e276f35f86b3980e088a95aa70c96dbddcdf658b
Author: Ismail Khoffi <[email protected]>
Date:   Tue Mar 19 14:36:42 2019 +0100

    remove 3421 from changelog

commit 8e62a3d62a5b04a5a7126d00d75925c990aec9c6
Author: Ismail Khoffi <[email protected]>
Date:   Tue Mar 19 12:19:02 2019 +0100

    Add #3421 to changelog and reorder alphabetically

commit 48aaccab8f3cc17032220a01a6d58acde695bbde
Merge: 551b6322 4162ebe8
Author: Ismail Khoffi <[email protected]>
Date:   Tue Mar 19 12:09:26 2019 +0100

    Merge in develop and update CHANGELOG.md

commit 4162ebe8b586deccc0e7476d8abafb75138bfe58
Author: Anton Kaliaev <[email protected]>
Date:   Tue Mar 19 11:38:32 2019 +0400

    types: refactor PB2TM.ConsensusParams to take BlockTimeIota as an arg (#3442)

    See https://github.com/tendermint/tendermint/pull/3403/files#r266208947

    In #3403 we unexposed BlockTimeIota from the ABCI, but it's still part
    of the ConsensusParams struct, so we have to remember to add it back
    after calling PB2TM.ConsensusParams. Instead, PB2TM.ConsensusParams
    should take it as an argument

    Fixes #3432

commit 551b6322f5a74f578a8487001e805e4e1da6394d
Author: Ethan Buchman <[email protected]>
Date:   Sat Mar 16 19:24:12 2019 -0400

    Update v0.31.0 release notes (#3434)

    * changelog: fix formatting

    * update release notes

    * update changelog

    * linkify

    * update UPGRADING

commit 52c4e15eb25e8a3d6e77f90239710de3655d4ec6
Author: Ismail Khoffi <[email protected]>
Date:   Thu Mar 14 16:07:06 2019 +0100

    changelog: more review fixes/release/v0.31.0 (#3427)

    * Update release summary

    * Add pubsub config changes

    * Add link to issue for pubsub changes

commit 5483ac6b0a2a75bc140487e8405392533a3f1687
Author: Ismail Khoffi <[email protected]>
Date:   Thu Mar 14 12:17:49 2019 +0100

    minor changes / fixes to release 0.31.0 (#3422)

    * bump ABCIVersion due to renaming BlockSizeParams -> BlockParams
    (https://github.com/tendermint/tendermint/pull/3417#discussion_r264974791)

    * Move changelog on consensus params entry to breaking

    * Add @melekes' suggestion for breaking change in pubsub into upgrading.md

    * Add changelog entry for #3351

    * Add changelog entry for #3358 & #3359

    * Add changelog entry for #3397

    * remove changelog entry for #3397 (was already released in 0.30.2)

    * move 3351 to improvements

    * Update changelog comment

commit 745713330736c5c751450245d88b8037cbee3aa6
Author: Anton Kaliaev <[email protected]>
Date:   Thu Mar 14 15:00:58 2019 +0400

    grpcdb: close Iterator/ReverseIterator after use (#3424)

    Fixes #3402

commit a59930a327065f2d091840586da6c35b6fd07472
Author: Anton Kaliaev <[email protected]>
Date:   Wed Mar 13 16:09:05 2019 +0400

    localnet: fix $LOG variable (#3423)

    Fixes #3421

    Before: it was creating a file named ${LOG:-tendermint.log} in .build/nodeX
    After: it creates a file named tendermint.log

commit 85c023db88a8517587f0e8499b1e89f0eea1112d
Author: Ismail Khoffi <[email protected]>
Date:   Tue Mar 12 20:07:26 2019 +0100

    Prep release v0.31.0:
     - update changelog, reset pending
     - bump versions
     - add external contributors (partly manually)

commit 4cbd36f341c8fee55a4ea386aee42a2c7fe9c67d
Merge: e42f833f 97681953
Author: Anton Kaliaev <[email protected]>
Date:   Tue Mar 12 21:22:03 2019 +0400

    Merge pull request #3415 from tendermint/master

    Merge master back to develop (do not squash)

commit e42f833fd4ce6cb886dbf594dcb2f3af61918325
Author: Anton Kaliaev <[email protected]>
Date:   Tue Mar 12 16:20:59 2019 +0400

    Merge master back to develop (#3412)

    * libs/db: close batch (#3397)

    ClevelDB requires closing when WriteBatch is no longer needed, https://godoc.org/github.com/jmhodges/levigo#WriteBatch.Close

    Fixes the memory leak in https://github.com/cosmos/cosmos-sdk/issues/3842

    * update changelog and bump version to 0.30.2

commit ad3e990c6a424d939b181a2a7c911eb4592c0c79
Author: Anton Kaliaev <[email protected]>
Date:   Mon Mar 11 23:59:00 2019 +0400

    fix GO_VERSION in installation scripts (#3411)

    there is no such file https://storage.googleapis.com/golang/go1.12.0.linux-amd64.tar.gz

    Fixes #3405

commit 676212fa8fdd1ddf29e9c352129c97da3c113769
Author: srmo <[email protected]>
Date:   Mon Mar 11 20:06:03 2019 +0100

    cmd: make sure to have 'testnet' create the data directory for nonvals (#3409)

    Fixes #3408

commit 303557203402f0a1b6db4a55cb2c6562765058b1
Author: Anton Kaliaev <[email protected]>
Date:   Mon Mar 11 22:52:09 2019 +0400

    cs: comment out log.Error to avoid TestReactorValidatorSetChanges timing out (#3401)

commit d741c7b4785c36cef4b0912900b6193db21d00e6
Author: Anton Kaliaev <[email protected]>
Date:   Mon Mar 11 22:45:58 2019 +0400

    limit number of /subscribe clients and queries per client (#3269)

    * limit number of /subscribe clients and queries per client

    Add the following config variables (under [rpc] section):
      * max_subscription_clients
      * max_subscriptions_per_client
      * timeout_broadcast_tx_commit

    Fixes #2826

    new HTTPClient interface for subscriptions

    finalize HTTPClient events interface

    remove EventSubscriber

    fix data race

    ```
    WARNING: DATA RACE
    Read at 0x00c000a36060 by goroutine 129:
      github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe.func1()
          /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:168 +0x1f0

    Previous write at 0x00c000a36060 by goroutine 132:
      github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe()
          /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:191 +0x4e0
      github.com/tendermint/tendermint/rpc/client.WaitForOneEvent()
          /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178
      github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1()
          /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298
      testing.tRunner()
          /usr/local/go/src/testing/testing.go:827 +0x162

    Goroutine 129 (running) created at:
      github.com/tendermint/tendermint/rpc/client.(*Local).Subscribe()
          /go/src/github.com/tendermint/tendermint/rpc/client/localclient.go:164 +0x4b7
      github.com/tendermint/tendermint/rpc/client.WaitForOneEvent()
          /go/src/github.com/tendermint/tendermint/rpc/client/helpers.go:64 +0x178
      github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync.func1()
          /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:139 +0x298
      testing.tRunner()
          /usr/local/go/src/testing/testing.go:827 +0x162

    Goroutine 132 (running) created at:
      testing.(*T).Run()
          /usr/local/go/src/testing/testing.go:878 +0x659
      github.com/tendermint/tendermint/rpc/client_test.TestTxEventsSentWithBroadcastTxSync()
          /go/src/github.com/tendermint/tendermint/rpc/client/event_test.go:119 +0x186
      testing.tRunner()
          /usr/local/go/src/testing/testing.go:827 +0x162
    ==================
    ```

    lite client works (tested manually)

    godoc comments

    httpclient: do not close the out channel

    use TimeoutBroadcastTxCommit

    no timeout for unsubscribe

    but 1s Local (5s HTTP) timeout for resubscribe

    format code

    change Subscribe#out cap to 1

    and replace config vars with RPCConfig

    TimeoutBroadcastTxCommit can't be greater than rpcserver.WriteTimeout

    rpc: Context as first parameter to all functions

    reformat code

    fixes after my own review

    fixes after Ethan's review

    add test stubs

    fix config.toml

    * fixes after manual testing

    - rpc: do not recommend to use BroadcastTxCommit because it's slow and wastes
    Tendermint resources (pubsub)
    - rpc: better error in Subscribe and BroadcastTxCommit
    - HTTPClient: do not resubscribe if err = ErrAlreadySubscribed

    * fixes after Ismail's review

    * Update rpc/grpc/grpc_test.go

    Co-Authored-By: melekes <[email protected]>

commit 15f621141dce76d992f6c8dcfbd4c522878b6108
Author: Anton Kaliaev <[email protected]>
Date:   Mon Mar 11 22:21:17 2019 +0400

    remove TimeIotaMs from ABCI consensus params (#3403)

    Also

    - init substructures to avoid panic in pb2tm.ConsensusParams
    Before: if csp.Block is nil and we later try to access/write to it,
    we'll panic.
    After: if csp.Block is nil and we later try to access/write to it,
    there'll be no panic.

commit dc359bd3a51c52803a6af820a36ee41796284e87
Author: Anca Zamfir <[email protected]>
Date:   Mon Mar 11 16:17:25 2019 +0200

    types: remove check for priority order of existing validators (#3407)

    When scaling and averaging is invoked, it is possible to have validators
    with close priorities ending up with same priority. With the current code,
    this  makes it impossible to verify the priority orders before and after updates.

    Fixes #3383

commit 976819537d199aa43c6ad55ec4e8958323322bb0
Merge: e0f89364 f996b10f
Author: Ethan Buchman <[email protected]>
Date:   Mon Mar 11 08:17:14 2019 -0400

    Merge pull request #3399 from tendermint/release/v0.30.2

    Release/v0.30.2

commit 100ff08de93ff1907bf810f584ec5bdc7a2a5260
Author: Anton Kaliaev <[email protected]>
Date:   Mon Mar 11 15:31:53 2019 +0400

    p2p: do not panic when filter times out (#3384)

    Fixes #3369

commit f996b10f479d7c9a6d81cca5a02c47b29a52b3f3
Author: Anton Kaliaev <[email protected]>
Date:   Sun Mar 10 13:06:34 2019 +0400

    update changelog and bump version to 0.30.2

commit 36d7180ca216f0d7ff62851fa441de2b0371d699
Author: Yumin Xia <[email protected]>
Date:   Sun Mar 10 00:46:32 2019 -0800

    libs/db: close batch (#3397)

    ClevelDB requires closing when WriteBatch is no longer needed, https://godoc.org/github.com/jmhodges/levigo#WriteBatch.Close

    Fixes the memory leak in https://github.com/cosmos/cosmos-sdk/issues/3842

commit b021f1e505482fb34f8a8e57cd86b171e4a57344
Author: Yumin Xia <[email protected]>
Date:   Sun Mar 10 00:46:32 2019 -0800

    libs/db: close batch (#3397)

    ClevelDB requires closing when WriteBatch is no longer needed, https://godoc.org/github.com/jmhodges/levigo#WriteBatch.Close

    Fixes the memory leak in https://github.com/cosmos/cosmos-sdk/issues/3842

commit 90794260bcd23989e4bd8530d596e39d9509e7ca
Author: mircea-c <[email protected]>
Date:   Sat Mar 9 10:13:36 2019 -0500

    circleci: removed complexity from docs deployment job  (#3396)

commit b6a510a3e7cefca56af4bdb009413e3950c6d59e
Author: Anton Kaliaev <[email protected]>
Date:   Fri Mar 8 09:46:09 2019 +0400

    make ineffassign linter pass (#3386)

    Refs #3262

    This fixes two small bugs:

    1) lite/dbprovider: return `ok` instead of true in parse* functions. It's weird that we're ignoring `ok` value before.
    2) consensus/state: previously because of the shadowing we almost never output "Error with msg". Now we declare both `added` and `err` in the beginning of the function, so there's no shadowing.

commit e415c326f93e5fa473d83a0ea6fa495ad0ea87f3
Author: Ismail Khoffi <[email protected]>
Date:   Fri Mar 8 06:40:59 2019 +0100

    update golang.org/x/crypto (#3392)

    Update Gopkg.lock via dep ensure --update golang.org/x/crypto

    see #3391 (comment) (nothing to review here really).

commit 28e9e9e7145b357cbd7078f4c83dcf66c13d4f7d
Author: Ismail Khoffi <[email protected]>
Date:   Thu Mar 7 17:03:57 2019 +0100

    update levigo to 1.0.0 (#3389)

    Although the version we were pinning to is from Nov. 2016 there were no substantial changes:

        jmhodges/levigo@2b8c778 added go-modules support (no code changes)
        jmhodges/levigo@853d788 added a badge to the readme

    closes #3381

commit 3ebfa99f2c2403c17348e41190406892abf21d2a
Author: Anton Kaliaev <[email protected]>
Date:   Thu Mar 7 19:35:04 2019 +0400

    do not pin repos without releases to exact revisions (#3382)

    We're pinning repos without releases because it's very easy to upgrade all the dependencies by executing dep ensure --upgrade. Instead, we should just never run this command directly, only dep ensure --upgrade <some repo>. And we can defend that in PRs.

    Refs #3374

    The problem with pinning to exact revisions: people who import Tendermint as a library (e.g. abci/types) are stuck with these revisions even though the code they import may not even use them.

commit 91b488f9a541f5dd69f0bb2db4249356733a2ed3
Author: YOSHIDA Masanori <[email protected]>
Date:   Thu Mar 7 22:02:13 2019 +0900

    docs: fix the reverse of meaning in spec (#3387)

    https://tools.ietf.org/html/rfc6962#section-2.1
    "The largest power of two less than the number of items" is actually correct!

        For n > 1, let k be the largest power of two smaller than n (i.e.,
        k < n <= 2k).

commit f25d727035f4615cbb4b43452191fd83218fc1e1
Author: Anton Kaliaev <[email protected]>
Date:   Thu Mar 7 09:10:34 2019 +0400

    make dupl linter pass (#3385)

    Refs #3262

commit 411bc5e49fcc3dda866ae799ba2ceda0084f80d4
Author: Anca Zamfir <[email protected]>
Date:   Wed Mar 6 09:54:49 2019 +0100

    types: followup after validator set changes (#3301)

    * fix failure in TestProposerFrequency

    * Add test to check priority order after updates

    * Changed applyRemovals() and removed Remove()

    Changed applyRemovals() similar to applyUpdates()
    Removed function Remove()
    Updated comments

    * review comments

    * simplify applyRemovals and add more comments

    * small correction in comment

    * Fix check in test

    * Fix priority check for centering, address review comments

    * fix assert for priority centering

    * review comments

    * review comments

    * cleanup and review comments

    added upper limit check for validator voting power
    moved check for empty validator set earlier
    moved panic on potential negative set length in verifyRemovals
    added more tests

    * review comments

commit 858875fbb8b94debc4973fdf5923ced6564d3c39
Author: Silas Davis <[email protected]>
Date:   Wed Mar 6 08:22:35 2019 +0000

    Copy secp256k1 code from go-ethereum to avoid GPL vendoring issues in (#3371)

    downstream

    Signed-off-by: Silas Davis <[email protected]>

commit 1eaa42cd25f2355f4b30440bddae435d056d1f44
Author: Ismail Khoffi <[email protected]>
Date:   Tue Mar 5 08:08:52 2019 +0100

    golang 1.12.0 (#3376)

     - update docker image on circleci
     - remove GOCACHE=off from Makefile (see:
     https://tip.golang.org/doc/go1.12#gocache)
     - update badge in readme
     - update in scripts/install
     - update Vagrantfile
     - update in networks/remote/integration.sh
     - tools/build/Makefile

commit 8c9df30e28a3b95efe537afe9930de2150999441
Author: Jack Zampolin <[email protected]>
Date:   Mon Mar 4 22:56:46 2019 -0800

    libs/db: Add cleveldb.Stats() (#3379)

    Fixes: #3378

    * Add stats to cleveldb implementation

    * update changelog

    * remote TODO

    also
    - sort keys
    - preallocate memory

    * fix const initializer []string literal is not a constant

    * add test

commit 52771e1287eb44eabae99e6d2781a57c577821bb
Author: Anton Kaliaev <[email protected]>
Date:   Mon Mar 4 13:24:44 2019 +0400

    make BlockTimeIota a consensus parameter, not a locally configurable … (#3048)

    * make BlockTimeIota a consensus parameter, not a locally configurable option

    Refs #2920

    * make TimeIota int64 ms

    Refs #2920

    * update Gopkg.toml

    * fixes after Ethan's review

    * fix TestRemoteSignerProposalSigningFailed

    * update changelog

commit f39138aa2e543438548a150bdad45304ccc3296b
Author: Anton Kaliaev <[email protected]>
Date:   Mon Mar 4 12:18:32 2019 +0400

    remove RoundState from EventDataRoundState (#3354)

    Before we're using it to get a round state in tests. Now it can be done
    by calling csX.GetRoundState. We will need to rewrite
    TestStateSlashingPrevotes and TestStateSlashingPrecommits, which are
    commented right now, to not rely on EventDataRoundState#RoundState
    field.

    Refs #1527

commit 8a962ffc4639eed95501114e0436ccffc52f8ef1
Author: Anton Kaliaev <[email protected]>
Date:   Mon Mar 4 12:17:38 2019 +0400

    deps: update gogo/protobuf from 1.1.1 to 1.2.1 and golang/protobuf from 1.1.0 to 1.3.0 (#3357)

    * deps: update gogo/proto from 1.1.1 to 1.2.1

    - verified changes manually
      git diff 636bf030~ ba06b47c --stat -- ':!*.pb.go' ':!test'

    * deps: update golang/protobuf from 1.1.0 to 1.3.0

    - verified changes manually
      git diff b4deda0~ c823c79 -- ':!*.pb.go' ':!test'

commit 3421e4dcd7c6937e9ae8f72af17b38840840a781
Author: zjubfd <[email protected]>
Date:   Sun Mar 3 05:36:52 2019 +0800

    make lightd availbe (#3364)

    1."abci_query": rpcserver.NewRPCFunc(c.ABCIQuery, "path,data,prove")
    "validators": rpcserver.NewRPCFunc(c.Validators, "height"),
    the parameters and function do not match, cause index out of range error.
    2. the prove of query is forced to be true, while default option is false.
    3. fix the wrong key of merkle

commit 976b1c2ef73e557a702ef17600e1eeaf4865f292
Author: zjubfd <[email protected]>
Date:   Sun Mar 3 04:21:21 2019 +0800

    fix pool timer leak bug, resolve#3353 (#3358)

    When remove peer, block pool simple remove bpPeer,
    but do not stop timer, that cause stopError for recorrected
    peers. Stop timer when remove from pool.

commit d95894152beb3b25423cdb109af006340f0ac80b
Author: zjubfd <[email protected]>
Date:   Sun Mar 3 04:17:37 2019 +0800

    fix dirty data in peerset,resolve #3304 (#3359)

    * fix dirty data in peerset

    startInitPeer before PeerSet add the peer,
    once mconnection start and Receive of one
    Reactor faild, will try to remove it from PeerSet
    while PeerSet still not contain the peer. Fix
    this by change the order.

    * fix test FilterDuplicate

    * fix start/stop race

    * fix err

commit 37a548414bbb5dc8dce9fe452ba5f933fad29684
Author: Sven-Hendrik Haase <[email protected]>
Date:   Sat Mar 2 07:06:57 2019 +0100

    docs: fix typo (#3373)

commit 853dd34d31bee6de593be94df87cbb508688249e
Author: Juan Leni <[email protected]>
Date:   Thu Feb 28 08:48:20 2019 +0100

    privval: improve Remote Signer implementation (#3351)

    This issue is related to #3107
    This is a first renaming/refactoring step before reworking and removing heartbeats.
    As discussed with @Liamsi , we preferred to go for a couple of independent and separate PRs to simplify review work.

    The changes:

        Help to clarify the relation between the validator and remote signer endpoints
        Differentiate between timeouts and deadlines
        Prepare to encapsulate networking related code behind RemoteSigner in the next PR

    My intention is to separate and encapsulate the "network related" code from the actual signer.

    SignerRemote ---(uses/contains)--> SignerValidatorEndpoint <--(connects to)--> SignerServiceEndpoint ---> SignerService (future.. not here yet but would like to decouple too)

    All reconnection/heartbeat/whatever code goes in the endpoints. Signer[Remote/Service] do not need to know about that.

    I agree Endpoint may not be the perfect name. I tried to find something "Go-ish" enough. It is a common name in go-kit, kubernetes, etc.

    Right now:
    SignerValidatorEndpoint:

        handles the listener
        contains SignerRemote
        Implements the PrivValidator interface
        connects and sets a connection object in a contained SignerRemote
        delegates PrivValidator some calls to SignerRemote which in turn uses the conn object that was set externally

    SignerRemote:

        Implements the PrivValidator interface
        read/writes from a connection object directly
        handles heartbeats

    SignerServiceEndpoint:

        Does most things in a single place
        delegates to a PrivValidator IIRC.

    * cleanup

    * Refactoring step 1

    * Refactoring step 2

    * move messages to another file

    * mark for future work / next steps

    * mark deprecated classes in docs

    * Fix linter problems

    * additional linter fixes

commit d6e2fb453d20841828cfbe1f6a7e8ca593c1b564
Author: Anton Kaliaev <[email protected]>
Date:   Thu Feb 28 11:31:59 2019 +0400

    update docs (#3349)

    * docs: explain create_empty_blocks configurations

    Closes #3307

    * Vagrantfile: install nodejs for docs

    * update docs instructions

    npm install does not make sense since there's no packages.json file

    * explain broadcast_tx_* tx format

    Closes #536

    * docs: explain how transaction ordering works

    Closes #2904

    * bring in consensus parameters explained

    * example for create_empty_blocks_interval

    * bring in explanation from https://github.com/tendermint/tendermint/issues/2487#issuecomment-424899799

    * link to formatting instead of duplicating info

commit ec9bff52346d9629f690147674326a41fcefd20f
Author: Anton Kaliaev <[email protected]>
Date:   Mon Feb 25 09:11:07 2019 +0400

    rename WAL#Flush to WAL#FlushAndSync (#3345)

    * rename WAL#Flush to WAL#FlushAndSync

    - rename auto#Flush to auto#FlushAndSync
    - cleanup WAL interface to not leak implementation details!
      * remove Group()
      * add WALReader interface and return it in SearchForEndHeight()
    - add interface assertions

    Refs #3337

    * replace WALReader with io.ReadCloser

commit 6797d8585167a9dc1e51843420ae4ab7eaf848a0
Author: Ismail Khoffi <[email protected]>
Date:   Mon Feb 25 06:06:21 2019 +0100

    p2p: fix comment in secret connection (#3348)

    Just a minor followup on the review if #3347: Fixes a comment. [#3347 (comment)](https://github.com/tendermint/tendermint/pull/3347#discussion_r259582330)

commit cdf3a74f4899b3c716703b932e682307752cd6b6
Author: Anton Kaliaev <[email protected]>
Date:   Sat Feb 23 19:48:28 2019 +0400

    Unclean shutdown on SIGINT / SIGTERM (#3308)

    * libs/common: TrapSignal accepts logger as a first parameter

     and does not block anymore
    * previously it was dumping "captured ..." msg to os.Stdout
    * TrapSignal should not be responsible for blocking thread of execution

    Refs #3238

    * exit with zero (0) code upon receiving SIGTERM/SIGINT

    Refs #3238

    * fix formatting in docs/app-dev/abci-cli.md

    Co-Authored-By: melekes <[email protected]>

    * fix formatting in docs/app-dev/abci-cli.md

    Co-Authored-By: melekes <[email protected]>

commit 41f91318e9e253a1b59cc1cc128ad412941b8e2a
Author: Anton Kaliaev <[email protected]>
Date:   Sat Feb 23 19:32:31 2019 +0400

    bound mempool memory usage (#3248)

    * bound mempool memory usage

    Closes #3079

    * rename SizeBytes to TxsTotalBytes

    and other small fixes after Zarko's review

    * rename MaxBytes to MaxTxsTotalBytes

    * make ErrMempoolIsFull more informative

    * expose mempool's txs_total_bytes via RPC

    * test full response

    * fixes after Ethan's review

    * config: rename mempool.size to mempool.max_txs

    https://github.com/tendermint/tendermint/pull/3248#discussion_r254034004

    * test more cases

    https://github.com/tendermint/tendermint/pull/3248#discussion_r254036532

    * simplify test

    * Revert "config: rename mempool.size to mempool.max_txs"

    This reverts commit 39bfa3696177aa46195000b90655419a975d6ff7.

    * rename count back to n_txs

    to make a change non-breaking

    * rename max_txs_total_bytes to max_txs_bytes

    * format code

    * fix TestWALPeriodicSync

    The test was sometimes failing due to processFlushTicks being called too
    early. The solution is to call wal#Start later in the test.

    * Apply suggestions from code review

commit e0adc5e807c8c2ea4ac03f8eeeb006b1a154d453
Author: Ismail Khoffi <[email protected]>
Date:   Sat Feb 23 16:25:57 2019 +0100

    secret connection check all zeroes (#3347)

    * reject the shared secret if is all zeros in case the blacklist was not
    sufficient

    * Add test that verifies lower order pub-keys are rejected at the DH step

    * Update changelog

    * fix typo in test-comment

commit 2137ecc130874118cc23995befed370958c34448
Author: Anton Kaliaev <[email protected]>
Date:   Sat Feb 23 08:11:27 2019 +0400

    pubsub 2.0 (#3227)

    * green pubsub tests :OK:

    * get rid of clientToQueryMap

    * Subscribe and SubscribeUnbuffered

    * start adapting other pkgs to new pubsub

    * nope

    * rename MsgAndTags to Message

    * remove TagMap

    it does not bring any additional benefits

    * bring back EventSubscriber

    * fix test

    * fix data race in TestStartNextHeightCorrectly

    ```
    Write at 0x00c0001c7418 by goroutine 796:
      github.com/tendermint/tendermint/consensus.TestStartNextHeightCorrectly()
          /go/src/github.com/tendermint/tendermint/consensus/state_test.go:1296 +0xad
      testing.tRunner()
          /usr/local/go/src/testing/testing.go:827 +0x162

    Previous read at 0x00c0001c7418 by goroutine 858:
      github.com/tendermint/tendermint/consensus.(*ConsensusState).addVote()
          /go/src/github.com/tendermint/tendermint/consensus/state.go:1631 +0x1366
      github.com/tendermint/tendermint/consensus.(*ConsensusState).tryAddVote()
          /go/src/github.com/tendermint/tendermint/consensus/state.go:1476 +0x8f
      github.com/tendermint/tendermint/consensus.(*ConsensusState).handleMsg()
          /go/src/github.com/tendermint/tendermint/consensus/state.go:667 +0xa1e
      github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
          /go/src/github.com/tendermint/tendermint/consensus/state.go:628 +0x794

    Goroutine 796 (running) created at:
      testing.(*T).Run()
          /usr/local/go/src/testing/testing.go:878 +0x659
      testing.runTests.func1()
          /usr/local/go/src/testing/testing.go:1119 +0xa8
      testing.tRunner()
          /usr/local/go/src/testing/testing.go:827 +0x162
      testing.runTests()
          /usr/local/go/src/testing/testing.go:1117 +0x4ee
      testing.(*M).Run()
          /usr/local/go/src/testing/testing.go:1034 +0x2ee
      main.main()
          _testmain.go:214 +0x332

    Goroutine 858 (running) created at:
      github.com/tendermint/tendermint/consensus.(*ConsensusState).startRoutines()
          /go/src/github.com/tendermint/tendermint/consensus/state.go:334 +0x221
      github.com/tendermint/tendermint/consensus.startTestRound()
          /go/src/github.com/tendermint/tendermint/consensus/common_test.go:122 +0x63
      github.com/tendermint/tendermint/consensus.TestStateFullRound1()
          /go/src/github.com/tendermint/tendermint/consensus/state_test.go:255 +0x397
      testing.tRunner()
          /usr/local/go/src/testing/testing.go:827 +0x162
    ```

    * fixes after my own review

    * fix formatting

    * wait 100ms before kicking a subscriber out

    + a test for indexer_service

    * fixes after my second review

    * no timeout

    * add changelog entries

    * fix merge conflicts

    * fix typos after Thane's review

    Co-Authored-By: melekes <[email protected]>

    * reformat code

    * rewrite indexer service in the attempt to fix failing test

    https://github.com/tendermint/tendermint/pull/3227/#issuecomment-462316527

    * Revert "rewrite indexer service in the attempt to fix failing test"

    This reverts commit 0d9107a098230de7138abb1c201877c246e89ed1.

    * another attempt to fix indexer

    * fixes after Ethan's review

    * use unbuffered channel when indexing transactions

    Refs https://github.com/tendermint/tendermint/pull/3227#discussion_r258786716

    * add a comment for EventBus#SubscribeUnbuffered

    * format code

commit 67fd42835448172d08616589f1473e7153cc0902
Author: Anton Kaliaev <[email protected]>
Date:   Fri Feb 22 11:49:08 2019 +0400

    fix TestWALPeriodicSync (#3342)

    The test was sometimes failing due to processFlushTicks being called too
    early. The solution is to call wal#Start later in the test.

commit f22ada442a249597dff765a59197bc57a8eae349
Author: Anton Kaliaev <[email protected]>
Date:   Fri Feb 22 11:48:40 2019 +0400

    refactor decideProposal in common_test (#3343)

commit ed1de135486608d8b4869760960e3eb35ade2c6d
Author: Anton Kaliaev <[email protected]>
Date:   Thu Feb 21 18:28:02 2019 +0400

    cs: update wal comments (#3334)

    * cs: update wal comments

    Follow-up to https://github.com/tendermint/tendermint/pull/3300

    * Update consensus/wal.go

    Co-Authored-By: melekes <[email protected]>

commit 4f83eec78251b9228f0edb76f261e3e387d21c12
Merge: db5d7602 e0f89364
Author: Ethan Buchman <[email protected]>
Date:   Wed Feb 20 10:11:33 2019 -0500

    Merge pull request #3339 from tendermint/master

    Merge master back to develop

commit e0f8936455029a40287a69d5b0e7baa4d5864da1
Merge: 28d75ec8 f2351dc7
Author: Ethan Buchman <[email protected]>
Date:   Wed Feb 20 10:10:49 2019 -0500

    Merge pull request #3326 from tendermint/release/v0.30.1

    Release/v0.30.1

commit f2351dc758e5c8cf6bc6c5c32931daa8ae6e49a9
Author: Ethan Buchman <[email protected]>
Date:   Wed Feb 20 10:05:57 2019 -0500

    update changelog

commit db5d7602fea6a3418d4ae83511c1a35cf9e1c9ca
Author: Daniil Lashin <[email protected]>
Date:   Wed Feb 20 14:34:52 2019 +0300

    docs: fix rpc Tx() method docs (#3331)

commit dff3deb2a95474ee1a10fc6aa7fd0a0a9a6aee94
Author: Thane Thomson <[email protected]>
Date:   Wed Feb 20 07:45:18 2019 +0200

    cs: sync WAL more frequently (#3300)

    As per #3043, this adds a ticker to sync the WAL every 2s while the WAL is running.

    * Flush WAL every 2s

    This adds a ticker that flushes the WAL every 2s while the WAL is
    running. This is related to #3043.

    * Fix spelling

    * Increase timeout to 2mins for slower build environments

    * Make WAL sync interval configurable

    * Add TODO to replace testChan with more comprehensive testBus

    * Remove extraneous debug statement

    * Remove testChan in favour of using system time

    As per
    https://github.com/tendermint/tendermint/pull/3300#discussion_r255886586,
    this removes the `testChan` WAL member and replaces the approach with a
    system time-oriented one. In this new approach, we keep track of the
    system time at which each flush and periodic flush successfully
    occurred.

    The naming of the various functions is also updated here to be more
    consistent with "flushing" as opposed to "sync'ing".

    * Update naming convention and ensure lock for timestamp update

    * Add Flush method as part of WAL interface

    Adds a `Flush` method as part of the WAL interface to enforce the idea
    that we can manually trigger a WAL flush from outside of the WAL. This
    is employed in the consensus state management to flush the WAL prior to
    signing votes/proposals, as per https://github.com/tendermint/tendermint/issues/3043#issuecomment-453853630

    * Update CHANGELOG_PENDING

    * Remove mutex approach and replace with DI

    The dependency injection approach to dealing with testing concerns could
    allow similar effects to some kind of "testing bus"-based approach. This
    commit introduces an example of this, where instead of relying on
    (potentially fragile) timing of things between the code and the test, we
    inject code into the function under test that can signal the test
    through a channel.

    This allows us to avoid the `time.Sleep()`-based approach previously
    employed.

    * Update comment on WAL flushing during vote signing

    Co-Authored-By: thanethomson <[email protected]>

    * Simplify flush interval definition

    Co-Authored-By: thanethomson <[email protected]>

    * Expand commentary on WAL disk flushing

    Co-Authored-By: thanethomson <[email protected]>

    * Add broken test to illustrate WAL sync test problem

    Removes test-related state (dependency injection code) from the WAL data
    structure and adds test code to illustrate the problem with using
    `WALGenerateNBlocks` and `wal.SearchForEndHeight` to test periodic
    sync'ing.

    * Fix test error messages

    * Use WAL group buffer size to check for flush

    A function is added to `libs/autofile/group.go#Group` in order to return
    the size of the buffered data (i.e. data that has not yet been flushed
    to disk). The test now checks that, prior to a `time.Sleep`, the group
    buffer has data in it. After the `time.Sleep` (during which time the
    periodic flush should have been called), the buffer should be empty.

    * Remove config root dir removal from #3291

    * Add godoc for NewWAL mentioning periodic sync

commit 9d4f59b8366ad393410051e09672b76578545d9e
Author: Anton Kaliaev <[email protected]>
Date:   Mon Feb 18 15:27:07 2019 +0400

    update changelog and bump version

commit d2c7f8dbcf238ccc3a2469e350523c7ed28be154
Author: Ismail Khoffi <[email protected]>
Date:   Mon Feb 18 11:08:22 2019 +0100

    p2p: check secret conn id matches dialed id (#3321)

    ref: [#3010 (comment)](https://github.com/tendermint/tendermint/issues/3010#issuecomment-464287627)

    > I tried searching for code where we authenticate a peer against its NetAddress.ID and couldn't find it. I don't see a reason to switch to Noise, but a need to ensure that the node's ID is authenticated e.g. after dialing from the address book.

    * p2p: check secret conn id matches dialed id

    * Fix all p2p tests & make code compile

    * add simple test for dialing with wrong ID

    * update changelog

    * address review comments

    * yet another place where to use IDAddressString and fix
    testSetupMultiplexTransport

commit 8283ca7ddb5b56090aaa15e005b40782fa618ae7
Author: Anton Kaliaev <[email protected]>
Date:   Mon Feb 18 13:23:40 2019 +0400

    3291 follow-up (#3323)

    * changelog: use issue number instead of PR number

    * follow up to #3291

    - rpc/test/helpers.go add StopTendermint(node) func
    - remove ensureDir(filepath.Dir(walFile), 0700)
    - mempool/mempool_test.go add type cleanupFunc func()

    * cmd/show_validator: wrap err to make it more clear

commit 59cc6d36c944012cd4ae39fbee698e8cf353005a
Author: Alessio Treglia <[email protected]>
Date:   Mon Feb 18 08:45:27 2019 +0100

    improve ResetTestRootWithChainID() concurrency safety (#3291)

    * improve ResetTestRootWithChainID() concurrency safety

    Rely on ioutil.TempDir() to create test root directories and ensure
    multiple same-chain id test cases can run in parallel.

    * Update config/toml.go

    Co-Authored-By: alessio <[email protected]>

    * clean up test directories after completion

    Closes: #1034

    * Remove redundant EnsureDir call

    * s/PanicSafety()/panic()/s

    * Put create dir functionality back in ResetTestRootWithChainID

    * Place test directories in OS's tempdir

    In modern UNIX and UNIX-like systems /tmp is very often
    mounted as tmpfs. This might speed test execution a bit.

    * Set 0700 to a const

    * rootsDirs -> configRootDirs

    * Don't double remove directories

    * Avoid global variables

    * Fix consensus tests

    * Reduce defer stack

    * Address review comments

    * Try to fix tests

    * Update CHANGELOG_PENDING.md

    Co-Authored-By: alessio <[email protected]>

    * Update consensus/common_test.go

    Co-Authored-By: alessio <[email protected]>

    * Update consensus/common_test.go

    Co-Authored-By: alessio <[email protected]>

commit af8793c01a168f0f15d3147673c572c0f8bf7863
Author: Zarko Milosevic <[email protected]>
Date:   Mon Feb 18 08:29:41 2019 +0100

    cs: reset triggered timeout precommit (#3310)

    * Reset TriggeredTimeoutPrecommit as part of updateToState

    * Add failing test and fix

    * fix DATA RACE in TestResetTimeoutPrecommitUponNewHeight

    ```
    WARNING: DATA RACE
    Read at 0x00c001691d28 by goroutine 691:
      github.com/tendermint/tendermint/consensus.decideProposal()
          /go/src/github.com/tendermint/tendermint/consensus/common_test.go:133 +0x121
      github.com/tendermint/tendermint/consensus.TestResetTimeoutPrecommitUponNewHeight()
          /go/src/github.com/tendermint/tendermint/consensus/state_test.go:1389 +0x958
      testing.tRunner()
       …
yutianwu pushed a commit to yutianwu/tendermint that referenced this pull request Jun 5, 2019
ClevelDB requires closing when WriteBatch is no longer needed, https://godoc.org/github.com/jmhodges/levigo#WriteBatch.Close

Fixes the memory leak in cosmos/cosmos-sdk#3842
yutianwu pushed a commit to yutianwu/tendermint that referenced this pull request Jun 5, 2019
mattkanwisher pushed a commit to loomnetwork/tendermint that referenced this pull request Jun 16, 2019
ClevelDB requires closing when WriteBatch is no longer needed, https://godoc.org/github.com/jmhodges/levigo#WriteBatch.Close

Fixes the memory leak in cosmos/cosmos-sdk#3842
mattkanwisher pushed a commit to loomnetwork/tendermint that referenced this pull request Jun 16, 2019
ClevelDB requires closing when WriteBatch is no longer needed, https://godoc.org/github.com/jmhodges/levigo#WriteBatch.Close

Fixes the memory leak in cosmos/cosmos-sdk#3842
enlight pushed a commit to loomnetwork/tendermint that referenced this pull request Jun 16, 2019
ClevelDB requires closing when WriteBatch is no longer needed, https://godoc.org/github.com/jmhodges/levigo#WriteBatch.Close

Fixes the memory leak in cosmos/cosmos-sdk#3842
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants