Parsing a flat list into a deep struct

[Bobini1]

Hi, sorry for starting another discussion so quickly but I've got an important question!

My previous question was about parsing a line until the end. This is an important feature of the file format I'm trying to read: one line = one "declaration".

It's a format for rhythm game charts (BMS). It goes something like this:

#TITLE example title
#ARTIST example artist

#00101:00110067
#00101:00220064
#00111:00000011
#00212:001167

The first two lines in this example are for metadata.
Line 4 defines two background sounds, at beat 2 and beat 4 of measure 001 (meter is 4/4 by default).
Line 5 does the same thing but with different sounds. This is legal, the sounds are supposed to play at the same time.
Line 6 defines a playable sound object at beat 4 on column 1 in measure 001.
Line 7 defines two playable sound objects at beats 2.33 and 3.66 on column 2 in measure 002.

What's hard about this:

Callbacks and sinks are cool but I've got little idea how to translate that to my model. It looks like this (simplified):

struct Measure
{
    static constexpr auto columnNumber = 9;
    std::array<std::vector<std::string>, columnNumber> playableNotes;
    std::vector<std::vector<std::string>> bgmNotes;
};

struct Tags
{
    std::optional<std::string> title;
    std::optional<std::string> artist;
    std::map<int64_t, Measure> measures;
};

As you can see, I've got a relatively deep structure here. as_aggregate was great for the title and artist but it's not going to work for the whole thing.

My current idea is to create a custom sink class with a ton of overloads with some tag type so that I can distinguish the data that comes in from every single line, one by one. State management would be very manual.

My question is, is this the right approach?

[foonathan]

I don't fully understand your grammar to give a concrete solution, but the general advice is as follows: if you're having nested structures, parse it with nested productions.

In your example, you have a Measure consisting of playable notes, and background notes. So add productions that parse playable notes and background notes and use them when parsing the measure. That way, you have the ability to add custom callbacks for the individual notes and can use a simpler callback for measure.

[Bobini1]

Custom callbacks for individual notes is what I wanted to do, yeah. I have that already. But a measure is a struct that's created from individual declarations that have that measure's number. They don't have to be in any order.
This is legal:

#99911:77
#03011:77
#TITLE interleaved stuff
#99912:66

(Measure 999, 030, the title tag, measure 999 again).

So I don't see an easy way to split it into separate productions. Everything is interleaved in a huge top-level list. I don't see a way to make a separate production for measure if it's like this.

Or is there a way?

[foonathan]

So you have a list of measures and attributes? dsl::list(dsl::p<measure> | dsl::p<tag>)

[Bobini1]

I have something like this, currently:

dsl::list(
   titleTag >> (dsl::member<& models::BmsChart::Tags::title> =
                  dsl::p<TextTag>) |
   artistTag >> (dsl::member<& models::BmsChart::Tags::artist> =
                   dsl::p<TextTag>) |
   bpmTag >> (dsl::member<& models::BmsChart::Tags::bpm> =
                dsl::p<FloatingPoint>)) +
 dsl::eof;

So you're right about the list. All those productions return strings.
If I wanted to parse a line with note data, the productions for those lines would all return something like:

std::pair<uint64_t, std::vector<std::string>>,

where the uint signifies the measure number (the period of time) this vector of notes belongs to. In the line

#00111:22,

001 is the measure number. 11 is the "channel", it's the type of objects that this line creates. It can be playable notes on some particular column, bgm sounds, bpm changes, background animation changes, etc., there is a ton of options. 22 is the identifier of the sound to play, in this case. Those identifiers are always two bytes long, usually base36.

If I had a separate production for each channel, they would return an std::pair like above.
But now that I think about it, I can have just one and separate it later. I didn't consider that.

So that single production would return something like

std::tuple<uint64_t, ChannelID, std::vector<std::string>>, I guess. I would keep a vector of those tuples in my struct.

struct Tags
{
    std::optional<std::string> title;
    std::optional<std::string> artist;
    std::vector<std::tuple<uint64_t, ChannelID, std::vector<std::string>>> measureBasedTags;
};

It would simplify things a bit. Can I easily either append to that vector (if my top level list encounters a tag which contains a measure number) or write to those optionals (when it encounters a metadata tag like #TITLE)?

[Bobini1]

In the end, I did it with a custom sink and it seems to work fine.

I used a lot of strong typedefs to distinguish between strings and I made one overload for tuple<measure, channel, vector>.

You can check it out here.

Thanks a lot for help!