Merge #4797 #4918

4797: Bump future from 0.18.2 to 0.18.3 in /doc/.sphinx r=newhoggy a=dependabot[bot]

Bumps [future](https://github.com/PythonCharmers/python-future) from 0.18.2 to 0.18.3.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/PythonCharmers/python-future/releases">future's releases</a>.</em></p>
<blockquote>
<h2>v0.18.3</h2>
<p>This is a minor bug-fix release containing a number of fixes:</p>
<ul>
<li>Backport fix for bpo-38804 (c91d70b)</li>
<li>Fix bug in fix_print.py fixer (dffc579)</li>
<li>Fix bug in fix_raise.py fixer (3401099)</li>
<li>Fix newint bool in py3 (fe645ba)</li>
<li>Fix bug in super() with metaclasses (6e27aac)</li>
<li>docs: fix simple typo, reqest -&gt; request (974eb1f)</li>
<li>Correct <strong>eq</strong> (c780bf5)</li>
<li>Pass if lint fails (2abe00d)</li>
<li>Update docker image and parcel out to constant variable.  Add comment to update version constant (45cf382)</li>
<li>fix order (f96a219)</li>
<li>Add flake8 to image (046ff18)</li>
<li>Make lint.sh executable (58cc984)</li>
<li>Add docker push to optimize CI (01e8440)</li>
<li>Build System (42b3025)</li>
<li>Add docs build status badge to README.md (3f40bd7)</li>
<li>Use same docs requirements in tox (18ecc5a)</li>
<li>Add docs/requirements.txt (5f9893f)</li>
<li>Add PY37_PLUS, PY38_PLUS, and PY39_PLUS (bee0247)</li>
<li>fix 2.6 test, better comment (ddedcb9)</li>
<li>fix 2.6 test (3f1ff7e)</li>
<li>remove nan test (4dbded1)</li>
<li>include list test values (e3f1a12)</li>
<li>fix other python2 test issues (c051026)</li>
<li>fix missing subTest (f006cad)</li>
<li>import from old imp library on older python versions (fc84fa8)</li>
<li>replace fstrings with format for python 3.4,3.5 (4a687ea)</li>
<li>minor style/spelling fixes (8302d8c)</li>
<li>improve cmp function, add unittest (0d95a40)</li>
<li>Pin typing==3.7.4.1 for Python 3.3 compatiblity (1a48f1b)</li>
<li>Fix various py26 unit test failures (9ca5a14)</li>
<li>Add initial contributing guide with docs build instruction (e55f915)</li>
<li>Add docs building to tox.ini (3ee9e7f)</li>
<li>Support NumPy's specialized int types in builtins.round (b4b54f0)</li>
<li>Added r&quot;&quot;&quot; to the docstring to avoid warnings in python3 (5f94572)</li>
<li>Add <strong>subclasscheck</strong> for past.types.basestring (c9bc0ff)</li>
<li>Correct example in README (681e78c)</li>
<li>Add simple documentation (6c6e3ae)</li>
<li>Add pre-commit hooks (a9c6a37)</li>
<li>Handling of <strong>next</strong> and next by future.utils.get_next was reversed (52b0ff9)</li>
<li>Add a test for our fix (461d77e)</li>
<li>Compare headers to correct definition of str (3eaa8fd)</li>
<li><a href="https://github-redirect.dependabot.com/PythonCharmers/python-future/issues/322">#322</a> Add support for negative ndigits in round; additionally, fixing a bug so that it handles passing in Decimal properly (a4911b9)</li>
<li>Add tkFileDialog to future.movers.tkinter (f6a6549)</li>
<li>Sort before comparing dicts in TestChainMap (6126997)</li>
<li>Fix typo (4dfa099)</li>
<li>Fix formatting in &quot;What's new&quot; (1663dfa)</li>
<li>Fix typo (4236061)</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/PythonCharmers/python-future/commit/af1db970b0879b59e7aeb798c27a623144561cff"><code>af1db97</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/PythonCharmers/python-future/issues/613">#613</a> from PythonCharmers/lwan/0.18.3-release</li>
<li><a href="https://github.com/PythonCharmers/python-future/commit/079ee9b75441d36447cec9981fa1b0032862f64d"><code>079ee9b</code></a> Prepare for 0.18.3 release</li>
<li><a href="https://github.com/PythonCharmers/python-future/commit/02f7a8143d5b68f50a1cca44d8f5a58c1925a515"><code>02f7a81</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/PythonCharmers/python-future/issues/610">#610</a> from wshanks/wshanks-patch-1</li>
<li><a href="https://github.com/PythonCharmers/python-future/commit/c91d70b34ef0402aef3e9d04364ba98509dca76f"><code>c91d70b</code></a> Backport fix for bpo-38804</li>
<li><a href="https://github.com/PythonCharmers/python-future/commit/80523f383fbba1c6de0551e19d0277e73e69573c"><code>80523f3</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/PythonCharmers/python-future/issues/569">#569</a> from jmadler/master</li>
<li><a href="https://github.com/PythonCharmers/python-future/commit/5e5af71549c7a7fa0e28f881046e081e231e455d"><code>5e5af71</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/PythonCharmers/python-future/issues/582">#582</a> from r3m0t/patch-6</li>
<li><a href="https://github.com/PythonCharmers/python-future/commit/17e4bbd7c676a9a8efd20601e51675c95f74b330"><code>17e4bbd</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/PythonCharmers/python-future/issues/596">#596</a> from abjonnes/fix-print-trailing-comma</li>
<li><a href="https://github.com/PythonCharmers/python-future/commit/1b427ba70191927706282840835e31ae0733ee7e"><code>1b427ba</code></a> Merge branch 'xZise-official-count' into master</li>
<li><a href="https://github.com/PythonCharmers/python-future/commit/c8eb497336c76d300c6753b47c7f5de505660d7a"><code>c8eb497</code></a> Merge branch 'official-count' of <a href="https://github.com/xZise/python-future">https://github.com/xZise/python-future</a> into ...</li>
<li><a href="https://github.com/PythonCharmers/python-future/commit/dffc579dbb7c882fc01fa0c0dfa6b59acef7827d"><code>dffc579</code></a> Fix bug in fix_print.py fixer</li>
<li>Additional commits viewable in <a href="https://github.com/PythonCharmers/python-future/compare/v0.18.2...v0.18.3">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=future&package-manager=pip&previous-version=0.18.2&new-version=0.18.3)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting ``@dependabot` rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- ``@dependabot` rebase` will rebase this PR
- ``@dependabot` recreate` will recreate this PR, overwriting any edits that have been made to it
- ``@dependabot` merge` will merge this PR after your CI passes on it
- ``@dependabot` squash and merge` will squash and merge this PR after your CI passes on it
- ``@dependabot` cancel merge` will cancel a previously requested merge and block automerging
- ``@dependabot` reopen` will reopen this PR if it is closed
- ``@dependabot` close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- ``@dependabot` ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` use these labels` will set the current labels as the default for future PRs for this repo and language
- ``@dependabot` use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- ``@dependabot` use these assignees` will set the current assignees as the default for future PRs for this repo and language
- ``@dependabot` use this milestone` will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/input-output-hk/cardano-node/network/alerts).

</details>

4918: Fix key non extended key for StakeExtendedVerificationKeyShelley_ed25519_bip32 envelope r=newhoggy a=newhoggy

* Fix key non extended key for `StakeExtendedVerificationKeyShelley_ed25519_bip32` envelope
* Add ability to generate golden files from test using `CREATE_GOLDEN_FILES=1`
* New `diffFileVsGoldenFile` function

Resolves #4914

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: John Ky <[email protected]>

cardano-api/src/Cardano/Api/DeserialiseAnyOf.hs

@@ -310,6 +310,7 @@ deserialiseAnyVerificationKeyTextEnvelope bs =
     [ FromSomeType (AsVerificationKey AsByronKey) AByronVerificationKey
     , FromSomeType (AsVerificationKey AsPaymentKey) APaymentVerificationKey
     , FromSomeType (AsVerificationKey AsPaymentExtendedKey) APaymentExtendedVerificationKey
+    , FromSomeType (AsVerificationKey AsStakeExtendedKey) AStakeExtendedVerificationKey
     , FromSomeType (AsVerificationKey AsGenesisUTxOKey) AGenesisUTxOVerificationKey
     , FromSomeType (AsVerificationKey AsGenesisExtendedKey) AGenesisExtendedVerificationKey
     ]

cardano-cli/test/Test/Golden/Key.hs

@@ -12,5 +12,10 @@ keyTests :: IO Bool
 keyTests =
   H.checkSequential
     $ H.Group "Key command group"
-        [ ("golden_KeyNonExtendedKey", Test.Golden.Key.NonExtendedKey.golden_KeyNonExtendedKey)
+        [ ( "golden_KeyNonExtendedKey_GenesisExtendedVerificationKey"
+          , Test.Golden.Key.NonExtendedKey.golden_KeyNonExtendedKey_GenesisExtendedVerificationKey
+          )
+        , ( "golden_KeyNonExtendedKey_StakeExtendedVerificationKeyShelley"
+          , Test.Golden.Key.NonExtendedKey.golden_KeyNonExtendedKey_StakeExtendedVerificationKeyShelley
+          )
         ]

cardano-cli/test/Test/Golden/Key/NonExtendedKey.hs

@@ -1,7 +1,8 @@
 {-# LANGUAGE OverloadedStrings #-}
 
 module Test.Golden.Key.NonExtendedKey
-  ( golden_KeyNonExtendedKey
+  ( golden_KeyNonExtendedKey_GenesisExtendedVerificationKey
+  , golden_KeyNonExtendedKey_StakeExtendedVerificationKeyShelley
   ) where
 
 import           Control.Monad (void)
@@ -10,30 +11,52 @@ import qualified Hedgehog.Extras.Test.Base as H
 import qualified Hedgehog.Extras.Test.File as H
 import           System.FilePath ((</>))
 import           Test.OptParse (execCardanoCLI, propertyOnce)
-import           Test.Utilities (diffVsGoldenFile)
+import           Test.Utilities (diffFileVsGoldenFile)
 
 {- HLINT ignore "Use camelCase" -}
 
 -- | Test that converting a @cardano-address@ Byron signing key yields the
 -- expected result.
-golden_KeyNonExtendedKey :: Property
-golden_KeyNonExtendedKey =
+golden_KeyNonExtendedKey_GenesisExtendedVerificationKey :: Property
+golden_KeyNonExtendedKey_GenesisExtendedVerificationKey =
   propertyOnce . H.moduleWorkspace "tmp" $ \tempDir -> do
     genesisVKeyFp <- H.note "test/data/golden/key/non-extended-keys/shelley.000.vkey"
-    nonExtendedGenesisVKeyFp <- H.note $ tempDir </> "non-extended-shelley.000.vkey"
+    nonExtendedFp <-  H.note "test/data/golden/key/non-extended-keys/non-extended-shelley.000.vkey"
+    outFp <- H.note $ tempDir </> "non-extended-shelley.000.vkey"
 
     H.assertFilesExist [genesisVKeyFp]
 
     -- Convert the `cardano-address` signing key
     void $ execCardanoCLI
       [ "key", "non-extended-key"
       , "--extended-verification-key-file", genesisVKeyFp
-      , "--verification-key-file", nonExtendedGenesisVKeyFp
+      , "--verification-key-file", outFp
       ]
 
     -- Check for existence of the converted signing key file
-    H.assertFilesExist [nonExtendedGenesisVKeyFp]
+    H.assertFilesExist [outFp]
 
-    contents <- H.readFile nonExtendedGenesisVKeyFp
+    diffFileVsGoldenFile outFp nonExtendedFp
 
-    diffVsGoldenFile contents nonExtendedGenesisVKeyFp
+-- | Test that converting a @cardano-address@ Byron signing key yields the
+-- expected result.
+golden_KeyNonExtendedKey_StakeExtendedVerificationKeyShelley :: Property
+golden_KeyNonExtendedKey_StakeExtendedVerificationKeyShelley =
+  propertyOnce . H.moduleWorkspace "tmp" $ \tempDir -> do
+    genesisVKeyFp <- H.note "test/data/golden/key/non-extended-keys/stake.000.vkey"
+    nonExtendedFp <-  H.note "test/data/golden/key/non-extended-keys/non-extended-stake.000.vkey"
+    outFp <- H.note $ tempDir </> "non-extended-stake.000.vkey"
+
+    H.assertFilesExist [genesisVKeyFp]
+
+    -- Convert the `cardano-address` signing key
+    void $ execCardanoCLI
+      [ "key", "non-extended-key"
+      , "--extended-verification-key-file", genesisVKeyFp
+      , "--verification-key-file", outFp
+      ]
+
+    -- Check for existence of the converted signing key file
+    H.assertFilesExist [outFp]
+
+    diffFileVsGoldenFile outFp nonExtendedFp

cardano-cli/test/Test/Utilities.hs

@@ -1,27 +1,87 @@
-module Test.Utilities (diffVsGoldenFile) where
+module Test.Utilities
+  ( diffVsGoldenFile,
+    diffFileVsGoldenFile,
+  ) where
 
-import           Cardano.Prelude (ConvertText (..))
+import           Cardano.Prelude (ConvertText (..), HasCallStack)
 
 import           Control.Monad.IO.Class (MonadIO (liftIO))
 import           Data.Algorithm.Diff (PolyDiff (Both), getGroupedDiff)
 import           Data.Algorithm.DiffOutput (ppDiff)
 import           GHC.Stack (callStack)
+import qualified GHC.Stack as GHC
 import           Hedgehog (MonadTest)
+import qualified Hedgehog.Extras.Test as H
 import           Hedgehog.Extras.Test.Base (failMessage)
+import qualified Hedgehog.Internal.Property as H
+import qualified System.Directory as IO
+import qualified System.Environment as IO
+import qualified System.IO.Unsafe as IO
 
+-- | Whether the test should create the golden files if the file does ont exist.
+createFiles :: Bool
+createFiles = IO.unsafePerformIO $ do
+  value <- IO.lookupEnv "CREATE_GOLDEN_FILES"
+  return $ value == Just "1"
+
+-- | Diff contents against the golden file.  If CREATE_GOLDEN_FILES environment is
+-- set to "1", then should the gold file not exist it would be created.
+--
+-- Set the environment variable when you intend to generate or re-generate the golden
+-- file for example when running the test for the first time or if the golden file
+-- genuinely needs to change.
+--
+-- To re-generate a golden file you must also delete the golden file because golden
+-- files are never overwritten.
+--
 -- TODO: Improve the help output by saying the difference of
 -- each input.
 diffVsGoldenFile
-  :: (MonadIO m, MonadTest m)
-  => String   -- ^ actual content
-  -> FilePath -- ^ reference file
+  :: HasCallStack
+  => (MonadIO m, MonadTest m)
+  => String   -- ^ Actual content
+  -> FilePath -- ^ Reference file
   -> m ()
-diffVsGoldenFile actualContent referenceFile =
-  do
-    referenceLines <- map toS . lines <$> liftIO (readFile referenceFile)
-    let difference = getGroupedDiff actualLines referenceLines
-    case difference of
-      [Both{}] -> pure ()
-      _        -> failMessage callStack $ ppDiff difference
+diffVsGoldenFile actualContent referenceFile = GHC.withFrozenCallStack $ do
+  fileExists <- liftIO $ IO.doesFileExist referenceFile
+
+  if fileExists
+    then do
+      referenceLines <- map toS . lines <$> H.readFile referenceFile
+      let difference = getGroupedDiff actualLines referenceLines
+      case difference of
+        [Both{}] -> pure ()
+        _        -> failMessage callStack $ ppDiff difference
+    else if createFiles
+      then do
+        -- CREATE_GOLDEN_FILES is set, so we create any golden files that don't
+        -- already exist.
+        H.note_ $ "Creating golden file " <> referenceFile
+        H.writeFile referenceFile actualContent
+      else do
+        H.note_ $ mconcat
+          [ "Golden file " <> referenceFile
+          , " does not exist.  To create, run with CREATE_GOLDEN_FILES=1"
+          ]
+        H.failure
   where
     actualLines = Prelude.lines actualContent
+
+-- | Diff file against the golden file.  If CREATE_GOLDEN_FILES environment is
+-- set to "1", then should the gold file not exist it would be created.
+--
+-- Set the environment variable when you intend to generate or re-generate the golden
+-- file for example when running the test for the first time or if the golden file
+-- genuinely needs to change.
+--
+-- To re-generate a golden file you must also delete the golden file because golden
+-- files are never overwritten.
+diffFileVsGoldenFile
+  :: HasCallStack
+  => (MonadIO m, MonadTest m)
+  => FilePath -- ^ Actual file
+  -> FilePath -- ^ Reference file
+  -> m ()
+diffFileVsGoldenFile actualFile referenceFile = GHC.withFrozenCallStack $ do
+  contents <- H.readFile actualFile
+  diffVsGoldenFile contents referenceFile

cardano-cli/test/data/golden/key/non-extended-keys/non-extended-shelley.000.vkey

@@ -0,0 +1,5 @@
+{
+    "type": "GenesisVerificationKey_ed25519",
+    "description": "",
+    "cborHex": "58200834b58f4bdda9522bb202af1f546db4cbbd94b068ae72c9fd96d9b55279edf0"
+}

cardano-cli/test/data/golden/key/non-extended-keys/non-extended-stake.000.vkey

@@ -0,0 +1,5 @@
+{
+    "type": "StakeVerificationKeyShelley_ed25519",
+    "description": "",
+    "cborHex": "58200f205175c0a47cba409c328f066e31ea4e81ef211f539c12b64b4b14e1d87188"
+}

cardano-cli/test/data/golden/key/non-extended-keys/stake.000.vkey

@@ -0,0 +1,5 @@
+{
+    "type": "StakeExtendedVerificationKeyShelley_ed25519_bip32",
+    "description": "",
+    "cborHex": "58400f205175c0a47cba409c328f066e31ea4e81ef211f539c12b64b4b14e1d87188a54f03c3edad073428f37dbdad714b7c07371ca19fe66c72d41fda23a81d8309"
+}

doc/.sphinx/requirements.txt

@@ -15,7 +15,7 @@ sphinxemoji==0.1.6
 sphinx_markdown_tables==0.0.15
 CommonMark==0.9.1
 docutils==0.16
-future==0.18.2
+future==0.18.3
 idna==2.9
 imagesize==1.2.0
 Jinja2==2.11.3