Functional style parsing for custom data members

[rbroggi]

Using the MACRO style would be nice if we could provide a custom lambda/method/fucntion pointer defining the serialization/deserialization for a class member on a class body. A good example of usage would be for serialization and deserialization of DateTime formats.

For example let's suppose I want the following type to be JSON serializable/deserializable:

class Booking {
  public:
    std::string getName();
    DateTime getDate();
  private:
    std::string _name;
    DateTime _date;
}

Let's suppose that the custom DateTime structure can be easily converted/represented by a std::string it would be very convenient to use the jsoncons macros to bind converters from and to string. E.g.:
Let's assume I have those two methods defined somewhere:

DataTime fromStringToDataTime(const std::string& str);
std::string fromDataTimeToString(const DataTime& dataTime);

I would like to be able to use one of the MACROS like the following snippet :

JSONCONS_ALL_FUNCTION_PARSER_TRAITS(Booking, 
  (getName, "name"),
  (getDate, fromStringToDataTime, fromDataTimeToString, "date")
)

More generally would be nice if the parameters could accept also lambdas:

JSONCONS_ALL_FUNCTION_PARSER_TRAITS(Booking, 
  (getName, "name"),
  (getDate, [](const std::string& str) { return Dat3Time(); }, [](const DateTime& date){ return std::string();}, "date")
)

It seems ambitious and I don't know the perks of the implementation but as user it would greatly benefit us.

More generally the functions / lambdas to be provided are constrained in a way that:

1. the first one should provide a conversion from a data-structure which jsoncons knows how to serialize/deserialize (even custom ones with other macros defined)
2. the second one converts from the unkown type to the known data type in point 1.

As another example let's say I want to deserialize the following json:

{
  "company": "CompanySA",
  "resources": [
    {
      "employee_name": "Rod",
      "employee_surname": "Bro"
    },
    {
      "employee_name": "Ale",
      "employee_surname": "Dha"     
    }
  ]
}

into the following data structure:

class Company {
  public:
    std::string getName();
   const std::vector<uint64_t> getIds();
  private:
    std::string _name;
    std::vector<uint64_t> _employeeIds;
}

and let's say that JSONCONS knows how to deserialize/deserialize the following data structure (proper MACRO provided in order to tell jsoncons how to do it):

class Employee {
  public:
    std::string getName();
    DateTime getSurname();
  private:
    std::string _name;
    std::string _surname;
}

JSONCONS_ALL_CTOR_GETTER_NAME_TRAITS(Employee,
                                      (getName, "employee_name"),
                                      (getSurname, "employee_surname")
                                     )

Let's say that I have two internal functions that are able to map between std::vector<Employee> and std::vector<uint64_t> like:

std::vector<Employee> fromIdsToEmployees(const std::vector<uint64_t>& ids);
std::vector<uint64_t> fromEmployeesToIds(const std::vector<Employee>& ids);

I would like to provide the following macro to deserialize from the initial json:

JSONCONS_ALL_FUNCTION_PARSER_TRAITS(Company, 
  (getName, "company"),
  (getIds, fromEmployeesToIds,  fromIdsToEmployees, "resources")
)

Another huge value for that is that it would provide the ability for users to include Data types of third part libraries into the class members, the user only need to 'translate' from/to json in a functional way... (hope it's clear :) )

So even if my code was not the one to define DateTime datastructure I could define the feasible conversion from and to it.
(If there is already something similar I would be very interested to know)

[danielaparker]

I had a quick look at json_traits_macros.hpp, and I think your proposal is doable, since based on the count of items inside the parentheses, we can have different macro expansions. Since we already have 27! convenience macros, I would prefer to allow additional parameters inside parentheses in the existing _NAME_ macros than to introduce new macro names.

I'll need to investigate further.

[rbroggi]

For me the _NAME_ convenience macro works just as well. I just used another name for the matter of the example.

Another interesting property of this feature would be to validate 'inline' or with custom functions/logics. E.g. even if my field is a std::string I might provide two std::function<std::string(std::string)> in the macro and validate my custom string format.
So let's say for instance I have a field which format should comply with a certain regular expression. I could have something like:

class Person {
  public:
    std::string getName();
    std::string getSocialSecurityNumber();
  private:
    std::string _name;
    std::string _socialSecurityNumber;
}

JSONCONS_ALL_FUNCTION_PARSER_TRAITS(Person, 
  (getName, "name"),
  (getSocialSecurityNumber, 
      [] (const std::string& iNonValidated) {
          std::regex myRegex(("[a-z]{1}[0-9]{10}"));
          if (not std::regex_match(iNonValidated, myRegex) ) {
              throw MyCustomException();
          }
          return iNonValidated;
      },  
      std::identity, 
      "_socialSecurityNumber"
   )
)

So in this case you don't even need to 'change' the type but you provide a local fast validation method.

Or maybe even more expressive:

class Person {
  public:
    std::string getName();
    std::optional<std::string> getSocialSecurityNumber();
  private:
    std::string _name;
    std::optional<std::string> _socialSecurityNumber;
}

JSONCONS_ALL_FUNCTION_PARSER_TRAITS(Person, 
  (getName, "name"),
  (getSocialSecurityNumber, 
      [] (const std::string& iNonValidated) {
          std::regex myRegex(("[a-z]{1}[0-9]{10}"));
          if (not std::regex_match(iNonValidated, myRegex) ) {
              return {};
          }
          return {iNonValidated};
      },  
      std::identity, 
      "_socialSecurityNumber"
   )
)

In this last example you can realize that if you want you could have unrelated std::functions. So you can relax a bit the properties of the functions to be passed to the macro:

1. First function: std::function<Type1(Type2)> needs to comply with:
   1. Type1 is the type of the member in the class;
   2. Type2 is a type that jsoncons knows how to deserialize;
2. Second function std::function<Type3(Type1)>:
   1. Type1 is the type of the member in the class;
   2. Type3 is a type that jsoncons knows how to serialize;

This could get even more interesting with the use of std::variant. Those are examples that I though of by reflecting just a bit on the functionality. I'm confident that there would be many other very interesting properties about this flexible design.
What do you think?

Rodrigo

[danielaparker]

The _NAME_ convenience macros now allow an optional mode parameter (JSONCONS_RDWR or JSONCONS_RDONLY) and three function objects, match (value matches expected), from (convert from type known to jsoncons) and into (convert into type known to jsoncons). jsoncons::always_true() can be used for the match argument. Square brackets indicate optionality.

JSONCONS_N_MEMBER_NAME_TRAITS(class_name,num_mandatory,
                              (member_name0,serialized_name0[,mode,match,from,into]),
                              (member_name1,serialized_name1[,mode,match,from,into])...) (5)

JSONCONS_ALL_MEMBER_NAME_TRAITS(class_name,
                                (member_name0,serialized_name0[,mode,match,from,into]),
                                (member_name1,serialized_name1[,mode,match,from,into])...) (6)

JSONCONS_TPL_N_MEMBER_NAME_TRAITS(num_template_params,
                                  class_name,num_mandatory,
                                  (member_name0,serialized_name0[,mode,match,from,into]),
                                  (member_name1,serialized_name1[,mode,match,from,into])...) (7)

JSONCONS_TPL_ALL_MEMBER_NAME_TRAITS(num_template_params,
                                    class_name,
                                    (member_name0,serialized_name0[,mode,match,from,into]),
                                    (member_name1,serialized_name1[,mode,match,from,into])...) (8)

JSONCONS_N_CTOR_GETTER_NAME_TRAITS(class_name,num_mandatory,
                                   (getter_name0,serialized_name0[,mode,match,from,into]),
                                   (getter_name1,serialized_name1[,mode,match,from,into])...) (15)

JSONCONS_ALL_CTOR_GETTER_NAME_TRAITS(class_name,
                                     (getter_name0,serialized_name0[,mode,match,from,into]),
                                     (getter_name1,serialized_name1[,mode,match,from,into])...) (16)

JSONCONS_TPL_N_CTOR_GETTER_NAME_TRAITS(num_template_params,
                                       class_name,num_mandatory,
                                       (getter_name0,serialized_name0[,mode,match,from,into]),
                                       (getter_name1,serialized_name1[,mode,match,from,into])...) (17)

JSONCONS_TPL_ALL_CTOR_GETTER_NAME_TRAITS(num_template_params,
                                         class_name,
                                         (getter_name0,serialized_name0[,mode,match,from,into]),
                                         (getter_name1,serialized_name1[,mode,match,from,into])...) (18)

JSONCONS_N_GETTER_SETTER_NAME_TRAITS(class_name,num_mandatory,
                                     (getter_name0,setter_name0,serialized_name0[,mode,match,from,into]),
                                     (getter_name1,setter_name1,serialized_name1[,mode,match,from,into])...) (23)

JSONCONS_ALL_GETTER_SETTER_NAME_TRAITS(class_name,
                                       (getter_name0,setter_name0,serialized_name0[,mode,match,from,into]),
                                       (getter_name1,setter_name1,serialized_name1[,mode,match,from,into])...) (24)

JSONCONS_TPL_N_GETTER_SETTER_NAME_TRAITS(num_template_params,
                                         class_name,num_mandatory,
                                         (getter_name0,setter_name0,serialized_name0[,mode,match,from,into]),
                                         (getter_name1,setter_name1,serialized_name1[,mode,match,from,into])...) (25)

JSONCONS_TPL_ALL_GETTER_SETTER_NAME_TRAITS(num_template_params,
                                           class_name,
                                           (getter_name0,setter_name0,serialized_name0[,mode,match,from,into]),
                                           (getter_name1,setter_name1,serialized_name1[,mode,match,from,into])...) (26)

Test cases may be found here.

Code is in 0.157.0.

Feedback appreciated.

[danielaparker]

I've moved the optional function objects to the end, which I think makes the documentation clearer, and edited my post above accordingly.

[rbroggi]

Will try to play around with it very soon :)

[danielaparker]

I've modified the macros so you can omit the second function object in your validation example.

[danielaparker]

The first validation example (decode_json throws) now looks like

#include <jsoncons/json.hpp>
#include <assert>

class Person 
{
      std::string name_;
      std::optional<std::string> socialSecurityNumber_;
  public:
      Person(const std::string& name, const std::optional<std::string>& socialSecurityNumber)
        : name_(name), socialSecurityNumber_(socialSecurityNumber)
      {
      }
      std::string getName() const
      {
          return name_;
      }
      std::optional<std::string> getSsn() const
      {
          return socialSecurityNumber_;
      }
};

JSONCONS_ALL_CTOR_GETTER_NAME_TRAITS(Person, 
  (getName, "name"),
  (getSsn, "social_security_number", 
      JSONCONS_RDWR, 
      [] (const jsoncooptional<std::string>& unvalidated) {
          if (!unvalidated)
          {
              return false;
          }
          std::regex myRegex("^(\\d{9})$");
          return std::regex_match(*unvalidated, myRegex);
      }
   )
)

using namespace jsoncons;

int main()
{
    std::vector<Person> persons1 = {Person("John Smith", "123456789"), Person("Jane Doe", "12345678")};    

    std::string output1;
    encode_json_pretty(persons1, output1);

    try
    {
        auto persons2 = decode_json<std::vector<Person>>(output1);
    }
    catch (const std::exception& e)
    {
        std::cout << e.what() << "\n\n";
    }
}

Output:

Not a Person

The second validation example (decode_json doesn't throw) looks like

#include <jsoncons/json.hpp>
#include <assert>

class Person 
{
      std::string name_;
      jsoncooptional<std::string> socialSecurityNumber_;
  public:
      Person(const std::string& name, const jsoncooptional<std::string>& socialSecurityNumber)
        : name_(name), socialSecurityNumber_(socialSecurityNumber)
      {
      }
      std::string getName() const
      {
          return name_;
      }
      jsoncooptional<std::string> getSsn() const
      {
          return socialSecurityNumber_;
      }
};

JSONCONS_ALL_CTOR_GETTER_NAME_TRAITS(Person, 
  (getName, "name"),
  (getSsn, "social_security_number", 
      JSONCONS_RDWR, jsoncoalways_true(),
      [] (const jsoncons::optional<std::string>& unvalidated) {
          if (!unvalidated)
          {
              return unvalidated;
          }
          std::regex myRegex(("^(\\d{9})$"));
          if (!std::regex_match(*unvalidated, myRegex) ) {
              return jsoncooptional<std::string>();
          }
          return unvalidated;
      }
   )
)

using namespace jsoncons;

int main()
{
        std::string input = R"(
[
    {
        "name": "John Smith",
        "social_security_number": "123456789"
    },
    {
        "name": "Jane Doe",
        "social_security_number": "12345678"
    }
]
    )";

        auto persons = decode_json<std::vector<Person>>(input);

        std::cout << "(1)\n";
        for (const auto& person : persons)
        {
            std::cout << person.getName() << ", " 
                      << (person.getSsn() ? *person.getSsn() : "n/a") << "\n";
        }
        std::cout << "\n";

        std::string output;
        encode_json_pretty(persons, output);
        std::cout << "(2)\n" << output << "\n";

    }
}

Output:

(1)
John Smith, 123456789
Jane Doe, n/a

(2)
[
    {
        "name": "John Smith",
        "social_security_number": "123456789"
    },
    {
        "name": "Jane Doe",
        "social_security_number": null
    }
]

[rbroggi]

Dear Daniel, thank you very much for your help. I'm sorry I didn't have time during the week to play with it. I still plan to heavily use this feature but it might be deferred a bit.

[danielaparker]

I've added optional mode and match parameters, see updated description above. These support polymorphism and variants for types that are only distinguished by the value of a read-only member, e.g. type() in the example below.

#include <jsoncons/json.hpp>

namespace ns {

    class Shape
    {
    public:
        virtual ~Shape() = default;
        virtual double area() const = 0;
    };
      
    class Rectangle : public Shape
    {
        double height_;
        double width_;
    public:
        Rectangle(double height, double width)
            : height_(height), width_(width)
        {
        }

        double height() const
        {
            return height_;
        }

        double width() const
        {
            return width_;
        }

        double area() const override
        {
            return height_ * width_;
        }

        const std::string type() const
        {
            return "rectangle";
        }
    };

    class Triangle : public Shape
    { 
        double height_;
        double width_;

    public:
        Triangle(double height, double width)
            : height_(height), width_(width)
        {
        }

        double height() const
        {
            return height_;
        }

        double width() const
        {
            return width_;
        }

        double area() const override
        {
            return (height_ * width_)/2.0;
        }

        std::string type() const
        {
            return "triangle";
        }
    };                 

    class Circle : public Shape
    { 
        double radius_;

    public:
        Circle(double radius)
            : radius_(radius)
        {
        }

        double radius() const
        {
            return radius_;
        }

        double area() const override
        {
            constexpr double pi = 3.14159265358979323846;
            return pi*radius_*radius_;
        }

        std::string type() const
        {
            return "circle";
        }
    };     
    
} // namespace ns            

JSONCONS_ALL_CTOR_GETTER_NAME_TRAITS(ns::Rectangle,
    (type,"type",JSONCONS_RDONLY,[](const std::string& type){return type == "rectangle";}),
    (height, "height", JSONCONS_RDWR),
    (width, "width")
)

JSONCONS_ALL_CTOR_GETTER_NAME_TRAITS(ns::Triangle,
    (type,"type", JSONCONS_RDONLY, [](const std::string& type){return type == "triangle";}),
    (height, "height"),
    (width, "width")
)

JSONCONS_ALL_CTOR_GETTER_NAME_TRAITS(ns::Circle,
    (type,"type", JSONCONS_RDONLY, [](const std::string& type){return type == "circle";}),
    (radius, "radius")
)

JSONCONS_POLYMORPHIC_TRAITS(ns::Shape,ns::Rectangle,ns::Triangle,ns::Circle)

using namespace jsoncons;

int main()
{
    std::string input = R"(
[
    {"type" : "rectangle", "width" : 2.0, "height" : 1.5 },
    {"type" : "triangle", "width" : 3.0, "height" : 2.0 },
    {"type" : "circle", "radius" : 1.0 }
]
    )";

    auto shapes = decode_json<std::vector<std::unique_ptr<ns::Shape>>>(input);
    REQUIRE(shapes.size() == 3);
    for (const auto& shape : shapes)
    {
        std::cout << "area: " << shape->area() << "\n";
    }
    std::string output;

    encode_json_pretty(shapes, output);
    std::cout << "\n" << output << "\n";
}

Output:

area: 3
area: 3
area: 3.14159

[
    {
        "height": 1.5,
        "type": "rectangle",
        "width": 2.0
    },
    {
        "height": 2.0,
        "type": "triangle",
        "width": 3.0
    },
    {
        "radius": 1.0,
        "type": "circle"
    }
]

[rbroggi]

Nice feature !

[danielaparker]

See also #277