No valid bundles were found in the data directory.

[jonathantittle]

PHP Version: 7.2.2

composer require paragonie/certainty:^1

When running a test to check the file path is valid using:

<?php
use ParagonIE\
{
    Certainty\RemoteFetch
};

$fetcher = (new RemoteFetch())->getLatestBundle();

echo $fetcher->getFilePath();

I'm met with:

Uncaught ParagonIE\Certainty\Exception\BundleException: No valid bundles were found in the data directory. in ../vendor/paragonie/certainty/src/Fetch.php:80

Looking to ./data I'm seeing:

ca-certs-backup-20180219145811.json
ca-certs.cache
ca-certs.json
cacert-2016-11-02.pem
cacert-2017-01-16.pem
cacert-2017-06-07.pem
cacert-2017-09-20.pem
cacert-2018-01-17.pem

Thus it looks like we just need an updated cacert-*.pem correct?

[paragonie-scott]

This is happening inconsistently on some of our VMs in house, too. It's definitely a bug, but I'm not certain where it lies yet. I'll release a fix as soon as I find it.

[paragonie-scott]

Can you reproduce this in v1.0.2? I think upgrading sodium_compat to v1.6.0 fixed the issue, but I might just be having difficulty reproducing it in our lab.

[gusbemacbe]

I applied all the codes, including the symlink and @jonathantittle's, nothing worked and the page stoped working.

Maybe your guide didn't tell which files to require.

[paragonie-scott]

If you're using Composer, you only need vendor/autoload.php.

If you're not using Composer, you take the responsibility of autoloading files, etc. into your own hands. Use Composer, please.

[gusbemacbe]

I use Composer. But I didn't copy the whole composer to my web site server because they are many files, therefore, I copied only the folder paragonie to the root of my web site server.

[paragonie-scott]

But I didn't copy the whole composer to my web site server because they are many files

That's almost certainly a bad idea.

• certainty/composer.json

  Lines 26 to 31 in 5b4d881

  	"require": {
  	"php": "^5.6|^7",
  	"guzzlehttp/guzzle": "^6",
  	"paragonie/constant_time_encoding": "^1|^2",
  	"paragonie/sodium_compat": "^1.6"
  	},

• https://github.com/guzzle/guzzle/blob/ba1bce30baab94dddde21c4ae93872b1762d39fa/composer.json#L15-L19

Have you tried rsync?

[gusbemacbe]

I have just updated, I have the folders guzzlehttp, paragonie and psr. Is it allright?

[gusbemacbe]

Wait, I am a bit confused, for the web site to work, do I have to copy the composer.json and the folder vendor to the web site root server?

[thomaskonrad]

I'm having the same issue on PHP 7.2.3.

[thomaskonrad]

It seems to fail when the Ed25519 signature is checked, in Fetch.php:

// If the SHA256 doesn't match, fail fast.
if ($validator::checkSha256Sum($bundle)) {
    /** @var bool $valid */
    $valid = true;
    if ($checkEd25519Signature) {
        $valid = $valid && $validator::checkEd25519Signature($bundle); // <-- here
    }
    if ($checkChronicle) {
        $valid = $valid && $validator::checkChronicleHash($bundle);
    }
    if ($valid) {
        return $bundle;
    }
}

$validator::checkEd25519Signature($bundle); returns false for every bundle in the data directory. I have these:

cacert-2016-11-02.pem
cacert-2017-01-18.pem
cacert-2017-06-07.pem
cacert-2017-09-20.pem
cacert-2018-01-17.pem
cacert-2018-03-07.pem
ca-certs.cache
ca-certs.json

The newest one has these signature values:

{
  "chronicle": "Jmto9HgxYETn-1JA6YVjDEs7OyjY_bffb2kfy-AGM2E=",
  "date": "2018-03-07",
  "file": "cacert-2018-03-07.pem",
  "sha256": "79ea479e9f329de7075c40154c591b51eb056d458bc4dff76d9a4b9c6c4f6d0b",
  "signature": "06dc96f0bc32ee82eb7611ac7fe0bfa646fd4139a65fe7999a404377e4b4a3272f74c509c1cbb1a6f509c8c7d438e79e95982b1f992c7fc6071d99e6f103680c"
}

By the way, I use Certainty via https://github.com/DivineOmega/password_exposed/.

[thomaskonrad]

By the way, I use libsodium 1.0.16. Let me know if you need anything to reproduce the issue.

[paragonie-scott]

Can you please clone Certainty, run composer install, drop this script in the same directory as composer.json, then run it?

<?php
require 'vendor/autoload.php';

$json = \json_decode(file_get_contents('data/ca-certs.json'), true);

$pk = hex2bin('98f2dfad4115fea9f096c35485b3bf20b06e94acac3b7acf6185aa5806020342');
foreach ($json as $row) {
    $miss = false;
    $file = $row['file'];
    $contents = \file_get_contents('data/' . $file);
    $sha256 = hex2bin($row['sha256']);
    $signature = hex2bin($row['signature']);

    if (\hash_file('sha256', 'data/' . $file, true) !== $sha256) {
        if (!$miss) {
            echo $file, PHP_EOL;
        }
        $miss = true;
        echo "\tHash mismatch (hash_file())!\n";
    }
    if (\hash('sha256', $contents, true) !== $sha256) {
        if (!$miss) {
            echo $file, PHP_EOL;
        }
        $miss = true;
        echo "\tHash mismatch (hash())!\n";
    }

    if (!\ParagonIE_Sodium_Compat::crypto_sign_verify_detached($signature, $contents, $pk)) {
        if (!$miss) {
            echo $file, PHP_EOL;
        }
        $miss = true;
        echo "\tSignature invalid (ParagonIE_Sodium_Compat)!\n";
    }
    if (!\ParagonIE_Sodium_File::verify($signature, 'data/' . $file, $pk)) {
        if (!$miss) {
            echo $file, PHP_EOL;
        }
        $miss = true;
        echo "\tSignature invalid (ParagonIE_Sodium_File)!\n";
    }
}

[thomaskonrad]

x:~/Development/certainty$ php test.php 
x:~/Development/certainty$ echo $?
0

[paragonie-scott]

This is as far as I was able to get with debugging. It's incredibly unlikely that the SHA256 hash would match, but suddenly the signature would fail, unless there was a bug in ParagonIE_Sodium_File. Yet when I invoke that separately, it doesn't fail.

Maybe I'll refactor Certainty to just load the whole file into memory and use ParagonIE_Sodium_Compat.

[paragonie-scott]

Can you pull dev-master, delete ca-certs.cache, and see if the "no valid bundles" error is resolved?

[thomaskonrad]

I can confirm that it works now.

By the way, I also get the warning Warning: Use of undefined constant CURLOPT_SSLVERSION - assumed 'CURLOPT_SSLVERSION' (this will throw an Error in a future version of PHP) (I already got that before).

[paragonie-scott]

Okay, so, I think the original issue is filesystem-related and probably a heisenbug.

CURLOPT_SSLVERSION should be defined. What version of cURL is installed? 80be5d5 should make the warning go away.

[paragonie-scott]

Releasing v1.0.3 with these fixes.

[thomaskonrad]

Awesome. Bug gone, warning gone. Thank you for the very quick response.

[paragonie-scott]

Happy to help. :)

Hopefully this bug is gone for everyone in v1.0.3. Please let me know if that's not the case.

[1337GameDev]

Hmm, this bug seems to be happening again :/

Seems to be caused by line 93 in Fetch.php (paragonie/certainty/src/Fetch.php):

$valid = $valid && $validator->checkChronicleHash($bundle);

The certs DO download (using v1.8.1 of sodium compat), but they fail this check. Any hints?

[paragonie-scott]

Update to the latest version of Certainty. Our public key was rotated.

[1337GameDev]

Oh, I thought I did fetch the latest, i'll run "composer update again"

Maybe a message could be added to detect this and give a warning in the log that the public key could rotate / an update might be needed.

How often should this be updated, before the key rotates again in the future? any way to fetch the key automatically? Or no?

[paragonie-scott]

Maybe a message could be added to detect this and give a warning in the log that the public key could rotate / an update might be needed.

There's no way to broadcast these messages. All the client-side software sees is "invalid public key" or "invalid signature". This exists to prevent downgrade attacks and subsequent social engineering.

How often should this be updated, before the key rotates again in the future? any way to fetch the key automatically? Or no?

Realistically, never. An error with our deployment script copied the key I created locally, so I had to rotate the key for the PHP community's Chronicle.

The most robust solution is to query a replica instead of the main instance, as documented here.

[1337GameDev]

I have the latest version and still get:

No valid bundles were found in the data directory.

[1337GameDev]

Seems to be the exact same failure as before:

Line 93 of Fetch.php:

$valid = $valid && $validator->checkChronicleHash($bundle);

[paragonie-scott]

It should be line 118. Make sure you're using version 2.x not 1.x (the latest is 2.6.0).

[1337GameDev]

I'm using certainty 2.3.2.

I am using PHPStorm, and have used "composer update" and their UI for managing composer dependencies and it hasn't found a newer version.

I'll use command line, and force it to update (using compsoer require) and report back.

[paragonie-scott]

Version 2.5 and older are expected to fail. 2.6 is the fixed version. https://github.com/paragonie/certainty/releases/tag/v2.6.0

[1337GameDev]

Yup. Everything seems to be working.

I thought I had the latest version this entire time. PHPStorm... you betray me.....

I really appreciate your help with this. Honestly the best support I've ever seen for an Open Source project.

If anybody else encounters this issue they can run this (in the root of their project with their composer json):

composer require "paragonie/certainty:2.6.0"

[paragonie-scott]

I really appreciate your help with this. Honestly the best support I've ever seen for an Open Source project.

I'm happy to hear this! We take bugs seriously (especially if they could imply security consequences on specific platforms). :)