Warn (don't fail!) on spelling errors. Fix typos reported by codespell. #13954

practicalswift · 2018-08-13T13:40:46Z

Check for common misspellings using codespell.
Fix recently introduced typos reported by codespell.

fanquake · 2018-08-13T14:07:33Z

Travis failure:

33.63s$ test/lint/lint-all.sh
test/lint/lint-all.sh: line 21: test/lint/lint-spelling.sh: Permission denied
^---- failure generated from test/lint/lint-spelling.sh

ken2812221 · 2018-08-13T15:10:54Z

.travis.yml

      cache: false
      language: python
      python: '3.6'
      install:
+        - travis_retry sudo apt-get install --no-install-recommends --no-upgrade -qq codespell


Can use

addons: apt: packages: - codespell

to avoid sudo

Oh that is much cleaner! Thanks!

DrahtBot · 2018-08-13T15:25:56Z

Note to reviewers: This pull request conflicts with the following ones:

Scripts and tools: gitian-build.py improvements and corrections #13998 (Scripts and tools: gitian-build.py improvements and corrections by hebasto)
Free BerkeleyEnvironment instances when not in use #11911 (Free CDBEnv instances when not in use by ryanofsky)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

laanwj · 2018-08-13T15:42:15Z

Does this mean we need to manually add words that are not recognized to the spell checker?
If we do this, I think this certainly needs maintenance instructions.

Empact · 2018-08-13T17:26:08Z

test/lint/lint-spelling.sh

+
+EXIT_CODE=0
+EXCLUDE_REGEXP="(src/example.cpp:.*example-false-positive-word) "
+SPELLING_ERRORS=$(codespell -d -q 3 $(git ls-files -- ":(exclude)src/leveldb/" ":(exclude)src/secp256k1/" ":(exclude)src/univalue/" ":(exclude)depends/" ":(exclude)doc/release-notes/" ":(exclude)src/qt/locale/" ":(exclude)test/functional/test_framework/__init__.py" ":(exclude)test/lint/lint-spelling.sh") | grep -vE "${EXCLUDE_REGEXP}")


nit: using the long form options here would be more self-documenting: --disable-colors --quiet-level=

Good point! Now fixed.

Empact · 2018-08-13T17:28:44Z

src/rpc/net.cpp

+            "\n" "Immediately disconnects from the specified peer node.\n"
+            "\n" "Strictly one out of 'address' and 'nodeid' can be provided to identify the node.\n"
+            "\n" "To disconnect by nodeid, either set 'address' to the empty string, or call using the named 'nodeid' argument only.\n"
+            "\n" "Arguments:\n"


Why are these changes necessary? There are plenty more leading \n in this file.

The old Ubuntu packaged version didn't understand \n and thus flagged "nTo" as a typo. That seems to have been fixed in the more recent pip installed version.

Empact · 2018-08-13T17:29:37Z

.travis.yml

+      addons:
+        apt:
+          packages:
+          - codespell


nit: codespell recommends installation via pip:
https://github.com/codespell-project/codespell

Good point! Now fixed

Empact · 2018-08-13T17:31:17Z

Concept ACK

practicalswift · 2018-08-13T21:19:04Z

@laanwj No not at all :-) codespell is based on a very comprehensive list of commonly misspelled words (see the codespell GitHub project for details). Words that are not recognised as known errors are simply ignored by codespell.

False positives – words recognised as errors by codespell but where we disagree with the classification for some reason – can be handled by simply adding them to test/lint/lint-spelling.ignore-words.txt.

To get rid of all false positives in the project only four entries are needed in the ignore file which is tiny considering that the Bitcoin project contains over 50 000 unique words :-)

$ wc -l < test/lint/lint-spelling.ignore-words.txt
4
$ git grep "" | tr A-Z a-z | sed "s/[^a-z]/ /g" | tr " " "\n" | grep "[a-z]" | sort | uniq | wc -l
55786

maflcko · 2018-08-13T22:15:39Z

src/qt/paymentserver.h

@@ -77,7 +77,7 @@ class PaymentServer : public QObject
    // to read from the file specified in the -rootcertificates setting,
    // or, if that's not set, to use the system default root certificates.
    // If you pass in a store, you should not X509_STORE_free it: it will be
-    // freed either at exit or when another set of CAs are loaded.
+    // freed either at exit or when another set of CA:s are loaded.


Yeah I would add these to test/lint/lint-spelling.ignore-words.txt rather than use : to separate them.

maflcko · 2018-08-13T22:16:49Z

src/qt/coincontroldialog.cpp

@@ -185,7 +185,7 @@ void CoinControlDialog::buttonSelectAllClicked()
                ui->treeWidget->topLevelItem(i)->setCheckState(COLUMN_CHECKBOX, state);
    ui->treeWidget->setEnabled(true);
    if (state == Qt::Unchecked)
-        coinControl()->UnSelectAll(); // just to be sure
+        coinControl()->DeSelectAll(); // just to be sure


Should probably be a scripted diff

Good point! Now fixed using a scripted-diff :-)

laanwj · 2018-08-14T07:19:14Z

I'm... not convinced about this.

I think running this once in a while is good, just to see if there's any really awkward misspellings, but running this in travis and failing the tests just because of a misspelled word in a comment seems overkill.

As I've said before, many times, linters should be added cautiously, there to prevent real problems resulting in bugs.

…ed by codespell. f8a81f7 lint: Add spell check linter (codespell) (practicalswift) ada3562 Fix typos reported by codespell (practicalswift) Pull request description: * Check for common misspellings using `codespell`. * Fix recently introduced typos reported by `codespell`. Tree-SHA512: 9974c0e640b411c7d0ebc5b45de253c19bac7fe3002cd98601ff8da8db584224c2fd7d331aee3df612c9f2cfef540d647a9b4c5a1a73fd208dc93ce4bf9e5e3e

laanwj · 2018-10-16T10:22:46Z

so in case anyone cares, i'm not happy about this

"can we add this?"
"well, "

I still strongly disagree with this, I don't think we should pile up linters for cosmetic issues.
The original idea of them was to flag serious issues that can result in bugs.
Failing the tests on some supposed spelling error in a comment really, goes too far.

"but… can we add it if it only warns?"

Yes, warning is ok with me

Though I think it would be best if one of us runs this periodically, for example for every major release, and fixes the jarring typos. I don't think this is something that necessarily needs to run continuously.

now, a few months later, it is causing Travis failures—apparently through #14179, without any discussion
this is not nice...

practicalswift · 2018-10-16T11:01:13Z

@laanwj Oh, I wasn't aware of the policy change introduced in #14179 so I don't have anything meaningful to add about that. Setting the discussion about that merge decision aside:

Do you have a link to the Travis failure? It would be interesting to see if it was a true positive or a false positive that caused codespell to complain.

luke-jr · 2018-10-16T11:08:55Z

It came up when someone tried to name a mutex mut. Seems unreasonable to demand variables have English word names...

laanwj · 2018-10-16T11:22:09Z

yes it came up on IRC @karel-3d

08:29 < karelb> I am trying to run my fork of bitcoind on travis.... and travis stops with this
08:29 < karelb> src/threadinterrupt.cpp:25: mut  ==> must, mutt, moot
08:29 < karelb> Warning: codespell identified likely spelling errors
08:29 < karelb> failure generated from test/lint/lint-spelling.sh
08:29 < karelb> .....ummmmm, ok?
08:30 < karelb> I did no change in threadinterrupt, and that "mut" is just some mutex
08:39 < karelb> well whatever, one commit that renamed "mut" to "mutex" fixed that, I just wonder why I needed to do that...
08:40 < gwillen> karelb: your travis instance is stupid
08:40 < gwillen> if you make it less stupid you will have fewer problems
08:40 < gwillen> it is trying to spell-correct your code, and doing it moronically
08:40 < karelb> :D
08:40 < gwillen> and seems to be set to warnings-as-errors.
08:40 < karelb> I use travis-ci.org
08:41 < gwillen> well, apparently they are stupid
08:41 < gwillen> but I imagine there is an option not to make codespell warnings errors
08:41 < karelb> the same config as bitcoin core
08:41 < gwillen> which they absolutely should not be
08:41 < gwillen> oh, hm
08:41 < gwillen> I mean, seemingly not
08:42 < karelb> well let's see what happens when I make a PR, if that `mut` stuff is still there
10:49 < wumpus> karelb: lol that's stupid
10:50 < wumpus> please don't tell me that stupid spelling check is mandatory in travis now

practicalswift · 2018-10-16T11:27:26Z

Oh, that sounds really strange.

I'm unable to reproduce locally:

$ grep mut src/threadinterrupt.cpp
        LOCK(mut);
    WAIT_LOCK(mut, lock);
$ codespell src/threadinterrupt.cpp
$ echo $?
0

Anyone who has been able to reproduce?

practicalswift · 2018-10-16T11:30:24Z

I was able to reproduce after upgrading codespell. Will submit a PR fixing this.

scravy · 2018-10-16T11:30:47Z

@practicalswift Which versions of codespell?

practicalswift · 2018-10-16T11:35:16Z

@scravy

$ pip3 install codespell --upgrade
Requirement already up-to-date: codespell in /usr/local/lib/python3.6/dist-packages
$ codespell --version
1.14.0

practicalswift · 2018-10-16T11:40:50Z

Fixed in #14495 :-)

practicalswift · 2018-10-16T11:59:00Z

@scravy Seems like this was introduced somewhere between 1.13.1 and 1.14.0. We're pinning to 1.13.0 in Travis. I don't know how a more recent got installed in @karel-3d:s Travis setup. @karel-3d, do you know? :-)

karelbilek · 2018-10-16T13:54:50Z

I cloned the repo, did some experimental changes, and used Travis-ci.org.

The build is here

https://travis-ci.org/karel-3d/bitcoin/builds/442019337

It's branched from this commit, 18 days old. (Maybe it will go away if I rebase on current master?)

0809e68

karelbilek · 2018-10-16T13:58:06Z

Oh yeah. It seems I have branched the master before this commit

d10f2cd

Which means that if I rebase to master, it would indeed go away.

practicalswift · 2018-10-16T14:13:51Z

@karel-3d Thanks for the clarification! That explains the mystery :-)

c32cf6a Add ignored word: mut (practicalswift) 4ae50da Revert "qa: Fix codespell error and have lint-spelling error instead of warn" (practicalswift) Pull request description: Revert `codespell` policy change introduced in #14179. Context: #13954 (comment) Tree-SHA512: 4606b19bb32cdd661f90b3778759818d3493e5ed1a4a2f95982f07eeb6b9c889bc8d53cde31706e0a3b9524c3d3a7378f1b769a60baeb0d00da4c68fd3068114

… reported by codespell. f8a81f7 lint: Add spell check linter (codespell) (practicalswift) ada3562 Fix typos reported by codespell (practicalswift) Pull request description: * Check for common misspellings using `codespell`. * Fix recently introduced typos reported by `codespell`. Tree-SHA512: 9974c0e640b411c7d0ebc5b45de253c19bac7fe3002cd98601ff8da8db584224c2fd7d331aee3df612c9f2cfef540d647a9b4c5a1a73fd208dc93ce4bf9e5e3e

c32cf6a Add ignored word: mut (practicalswift) 4ae50da Revert "qa: Fix codespell error and have lint-spelling error instead of warn" (practicalswift) Pull request description: Revert `codespell` policy change introduced in bitcoin#14179. Context: bitcoin#13954 (comment) Tree-SHA512: 4606b19bb32cdd661f90b3778759818d3493e5ed1a4a2f95982f07eeb6b9c889bc8d53cde31706e0a3b9524c3d3a7378f1b769a60baeb0d00da4c68fd3068114

… reported by codespell. f8a81f7 lint: Add spell check linter (codespell) (practicalswift) ada3562 Fix typos reported by codespell (practicalswift) Pull request description: * Check for common misspellings using `codespell`. * Fix recently introduced typos reported by `codespell`. Tree-SHA512: 9974c0e640b411c7d0ebc5b45de253c19bac7fe3002cd98601ff8da8db584224c2fd7d331aee3df612c9f2cfef540d647a9b4c5a1a73fd208dc93ce4bf9e5e3e

c32cf6a Add ignored word: mut (practicalswift) 4ae50da Revert "qa: Fix codespell error and have lint-spelling error instead of warn" (practicalswift) Pull request description: Revert `codespell` policy change introduced in bitcoin#14179. Context: bitcoin#13954 (comment) Tree-SHA512: 4606b19bb32cdd661f90b3778759818d3493e5ed1a4a2f95982f07eeb6b9c889bc8d53cde31706e0a3b9524c3d3a7378f1b769a60baeb0d00da4c68fd3068114

… reported by codespell. f8a81f7 lint: Add spell check linter (codespell) (practicalswift) ada3562 Fix typos reported by codespell (practicalswift) Pull request description: * Check for common misspellings using `codespell`. * Fix recently introduced typos reported by `codespell`. Tree-SHA512: 9974c0e640b411c7d0ebc5b45de253c19bac7fe3002cd98601ff8da8db584224c2fd7d331aee3df612c9f2cfef540d647a9b4c5a1a73fd208dc93ce4bf9e5e3e

c32cf6a Add ignored word: mut (practicalswift) 4ae50da Revert "qa: Fix codespell error and have lint-spelling error instead of warn" (practicalswift) Pull request description: Revert `codespell` policy change introduced in bitcoin#14179. Context: bitcoin#13954 (comment) Tree-SHA512: 4606b19bb32cdd661f90b3778759818d3493e5ed1a4a2f95982f07eeb6b9c889bc8d53cde31706e0a3b9524c3d3a7378f1b769a60baeb0d00da4c68fd3068114

… reported by codespell. f8a81f7 lint: Add spell check linter (codespell) (practicalswift) ada3562 Fix typos reported by codespell (practicalswift) Pull request description: * Check for common misspellings using `codespell`. * Fix recently introduced typos reported by `codespell`. Tree-SHA512: 9974c0e640b411c7d0ebc5b45de253c19bac7fe3002cd98601ff8da8db584224c2fd7d331aee3df612c9f2cfef540d647a9b4c5a1a73fd208dc93ce4bf9e5e3e

c32cf6a Add ignored word: mut (practicalswift) 4ae50da Revert "qa: Fix codespell error and have lint-spelling error instead of warn" (practicalswift) Pull request description: Revert `codespell` policy change introduced in bitcoin#14179. Context: bitcoin#13954 (comment) Tree-SHA512: 4606b19bb32cdd661f90b3778759818d3493e5ed1a4a2f95982f07eeb6b9c889bc8d53cde31706e0a3b9524c3d3a7378f1b769a60baeb0d00da4c68fd3068114

… reported by codespell. f8a81f7 lint: Add spell check linter (codespell) (practicalswift) ada3562 Fix typos reported by codespell (practicalswift) Pull request description: * Check for common misspellings using `codespell`. * Fix recently introduced typos reported by `codespell`. Tree-SHA512: 9974c0e640b411c7d0ebc5b45de253c19bac7fe3002cd98601ff8da8db584224c2fd7d331aee3df612c9f2cfef540d647a9b4c5a1a73fd208dc93ce4bf9e5e3e

fanquake added the Tests label Aug 13, 2018

practicalswift force-pushed the lint-spell branch 2 times, most recently from ab46eb5 to 9d8dd3d Compare August 13, 2018 13:46

practicalswift force-pushed the lint-spell branch 7 times, most recently from ad8c1bb to c505b76 Compare August 13, 2018 15:10

ken2812221 reviewed Aug 13, 2018

View reviewed changes

practicalswift force-pushed the lint-spell branch from b2bcf1b to ab54964 Compare August 13, 2018 15:21

DrahtBot mentioned this pull request Aug 13, 2018

travis: move script sections to files in .travis/ subject to shellcheck #13863

Merged

Empact reviewed Aug 13, 2018

View reviewed changes

practicalswift force-pushed the lint-spell branch 4 times, most recently from 10dd66a to d8725c0 Compare August 13, 2018 21:01

practicalswift changed the title ~~lint: Add spell check linter (codespell). Fix typos reported by codespell.~~ lint: Check for common misspellings using codespell. Fix typos reported by codespell. Aug 13, 2018

maflcko reviewed Aug 13, 2018

View reviewed changes

DrahtBot mentioned this pull request Aug 14, 2018

~~Correctly~~ terminate HTTP server #13501

Closed

maflcko merged commit f8a81f7 into bitcoin:master Sep 5, 2018

Bushstar mentioned this pull request Oct 11, 2018

Updates from bitcoin/master FeatherCoin/Feathercoin#366

Merged

practicalswift mentioned this pull request Oct 16, 2018

build: Warn (don't fail!) on spelling errors #14495

Merged

practicalswift deleted the lint-spell branch April 10, 2021 19:35

rht mentioned this pull request May 19, 2021

qa: Fix typos dogecoin/dogecoin#1965

Merged

bitcoin locked as resolved and limited conversation to collaborators Aug 18, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Warn (don't fail!) on spelling errors. Fix typos reported by codespell. #13954

Warn (don't fail!) on spelling errors. Fix typos reported by codespell. #13954

practicalswift commented Aug 13, 2018 •

edited

Loading

fanquake commented Aug 13, 2018

ken2812221 Aug 13, 2018

practicalswift Aug 13, 2018

DrahtBot commented Aug 13, 2018 •

edited

Loading

laanwj commented Aug 13, 2018

Empact Aug 13, 2018

practicalswift Aug 13, 2018

Empact Aug 13, 2018

practicalswift Aug 13, 2018

Empact Aug 13, 2018 •

edited

Loading

practicalswift Aug 13, 2018

Empact commented Aug 13, 2018

practicalswift commented Aug 13, 2018 •

edited

Loading

maflcko Aug 13, 2018

Empact Aug 14, 2018

maflcko Aug 13, 2018

practicalswift Aug 14, 2018 •

edited

Loading

laanwj commented Aug 14, 2018

laanwj commented Oct 16, 2018

practicalswift commented Oct 16, 2018 •

edited

Loading

luke-jr commented Oct 16, 2018

laanwj commented Oct 16, 2018

practicalswift commented Oct 16, 2018 •

edited

Loading

practicalswift commented Oct 16, 2018

scravy commented Oct 16, 2018

practicalswift commented Oct 16, 2018

practicalswift commented Oct 16, 2018

practicalswift commented Oct 16, 2018 •

edited

Loading

karelbilek commented Oct 16, 2018

karelbilek commented Oct 16, 2018

practicalswift commented Oct 16, 2018

Warn (don't fail!) on spelling errors. Fix typos reported by codespell. #13954

Warn (don't fail!) on spelling errors. Fix typos reported by codespell. #13954

Conversation

practicalswift commented Aug 13, 2018 • edited Loading

fanquake commented Aug 13, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DrahtBot commented Aug 13, 2018 • edited Loading

laanwj commented Aug 13, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Empact Aug 13, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Empact commented Aug 13, 2018

practicalswift commented Aug 13, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

practicalswift Aug 14, 2018 • edited Loading

Choose a reason for hiding this comment

laanwj commented Aug 14, 2018

laanwj commented Oct 16, 2018

practicalswift commented Oct 16, 2018 • edited Loading

luke-jr commented Oct 16, 2018

laanwj commented Oct 16, 2018

practicalswift commented Oct 16, 2018 • edited Loading

practicalswift commented Oct 16, 2018

scravy commented Oct 16, 2018

practicalswift commented Oct 16, 2018

practicalswift commented Oct 16, 2018

practicalswift commented Oct 16, 2018 • edited Loading

karelbilek commented Oct 16, 2018

karelbilek commented Oct 16, 2018

practicalswift commented Oct 16, 2018

practicalswift commented Aug 13, 2018 •

edited

Loading

DrahtBot commented Aug 13, 2018 •

edited

Loading

Empact Aug 13, 2018 •

edited

Loading

practicalswift commented Aug 13, 2018 •

edited

Loading

practicalswift Aug 14, 2018 •

edited

Loading

practicalswift commented Oct 16, 2018 •

edited

Loading

practicalswift commented Oct 16, 2018 •

edited

Loading

practicalswift commented Oct 16, 2018 •

edited

Loading