3 bit for "l" integer ubjson?

[ghost]

Hi,

I have adopted one of ubjson examples to just store a large vector of doubles. Unfortunately I get an exemption for file position 14 when the vector size is > 2^24 (16777216). Up to that number the example works fine.

If I got it wrong any help is much appreciated.

#include <cassert>
#include <iostream>
#include <fstream>
#include <vector>
#include <jsoncons/json.hpp>
#include <jsoncons_ext/ubjson/ubjson.hpp>

namespace ns {
    class hiking_reputon
    {
        std::vector<double> x_;
    public:
        hiking_reputon(std::vector<double> x)
            : x_(x)
        {
        }
        const std::vector<double>& x() const {return x_;}
        friend bool operator==(const hiking_reputon& lhs, const hiking_reputon& rhs)
        {
            return lhs.x_ == rhs.x_;
        }
        friend bool operator!=(const hiking_reputon& lhs, const hiking_reputon& rhs)
        {
            return !(lhs == rhs);
        };
    };
} // namespace ns

// Declare the traits. Specify which data members need to be serialized.
JSONCONS_ALL_CTOR_GETTER_TRAITS(ns::hiking_reputon, x)

int main()
{
  std::vector<double> x;x.resize(16777217);
  for(int i=0;i<x.size();++i){x[i]=(double)i;}
  ns::hiking_reputon val(x);

    // Encode a ns::hiking_reputation value to UBJSON
    std::vector<uint8_t> data;
    jsoncons::ubjson::encode_ubjson(val, data);
    std::ofstream xx("data.bin",std::ios::out | std::ios::binary);
    for(int i=0;i<data.size();++i){
      xx.write((char *) data.data()+i,sizeof(uint8_t));
    }
    xx.close();
    // Decode UBJSON to a ns::hiking_reputation value
    std::cout<<"start reading"<<std::endl;
    std::ifstream xin2("data.bin",std::ios::in | std::ios::binary);
    ns::hiking_reputon val2 = jsoncons::ubjson::decode_ubjson<ns::hiking_reputon>(xin2);

    assert(val2 == val);
}

[danielaparker]

The issue here is that by default jsoncons limits the maximum size of a UBJSON container, to 1 << 24 items, and raises an error if that limit is exceeded. The problem is that UBJSON is more susceptible than other binary formats to attacks, and we needed to set a limit when parsing to resolve a Google fuzzer issue. To read this file you'll need to set a larger max_items in ubjson_options,

jsoncons::ubjson::ubjson_options options;
options.max_items(std::numeric_limits<int32_t>::max());
ns::hiking_reputon val2 = jsoncons::ubjson::decode_ubjson<ns::hiking_reputon>(data, options);

That said, the exception that was being thrown

"Unable to convert into the provided type"

was distinctly unhelpful. This has been changed (currently on master) to

"Number of items in UBJSON object or array exceeds limit set in options"