creating nested structs

[michbens]

Hello,
Please consider the following schema:

struct S1
{
    s1f1:uint32;
    s1f2:uint32;
}

struct S2
{
    s2f1:uint32;
    s2f2:uint32;
}

struct S1S2
{
    field1: S1;
    field2: S2;
}

I tried the following to build:

ns_S1_ref_t field1 = ns_S1_create(builder, 1, 2);

ns_S2_ref_t field2 = ns_S1_create(builder, 3, 4);

ns_S1S2_ref_t s12 = ns_S1S2_create(builder, field1, field2);

I got the following error:

 error: too few arguments to function call, expected 5, have 3
    ns_S1S2_ref_t s12 = ns_S1S2_create(builder, field1, field2);
                        ~~~~~~~~~~~~~~

If I defined tables instead of structs it works.
It seems ns_S1S2_create expects all the nested structs fields.
How can I create nested structs by creating each structs and nested structs separately ?

Thanks

[mikkelfj]

The first thing to realize is that structs are self-contained blocks of memory, including nested structs, while tables are not. So you can't really first create the inner structs and then add a reference.

Have a look at builder.md
https://github.com/dvidelabs/flatcc/blob/master/doc/builder.md#structs

You can also look at the generated code because constructors are generated in an easily readable function as I recall.

I think the choice was to have all arguments flattened when using the constructor method because - why would you construct some fields but then not the children? You cannot first use child constructors because as mentioned, everything is created at once. So you can use the constructor if you provide all the child fields also in the constructor call as you suggest, but want to avoid.

The other option is to manually create the structs in native C memory and fill in the structs there, then add the outermost struct by pointer. In this case the struct will be efficiently copied to the output buffer on little endian systems, and on big endian systems the content will be endian converted. Be sure to null the struct with memset first to avoid garbage in padding areas. Failing to do so is not catastrophic, but bad style.

A third option is to get a temp pointer to a struct from the builder, for example via a push or add call - see builder.md. Here it is not necessary to do memset the struct.

Let me know if this works out for you and please ask again if you need more help.

BTW: it might be possible to use a constructor on a struct on the C stack before being added the builder. I'm not quite sure. But if it is, you should be able to use constructors also for child structs. You can look at the generated code.

Mikkel

[michbens]

OK, I tried this:

ns_S1_t S1s = {1, 2};
ns_S2_t S2s = {3, 4};
ns_S1S2_t son1 = {S1s, S2s};

And then

ns_S1S2_field1_start(B);

But ns_S1S2_field1_start() doesn't exists. I get a link error.
I don't understand how can I add son1 to binary ?

[mikkelfj]

Also, if you want to add the struct to a table, the call you are using will not work because it creates a separate memory block for the struct similar to child tables. But structs are embedded inside the block of memory that represents a table, i.e. structs are usually inline.

The call you are using can be used when structs are top level objects in a buffer (only supported by FlatCC), or when structs are used in unions. Otherwise you should a table field specific add or create call. In your case add if you want to add fields manually.

[mikkelfj]

But ns_S1S2_field1_start() doesn't exists. I get a link error.

This only works for tables. If S1S2 were a table with a member field1 of type S1, then something along those lines should work.

You need to figure out what kind of buffer you will be creating.

Do you want to create a FlatBuffer with a struct as root?

There should be something like S1S2_start_as_root();

If you do this, you must be aware that the buffer will not be readable by other languages because flatc chose not to support structs as root objects at some point.

[mikkelfj]

I earlier wrote that there might be a struct constructor for structs in C memory. This method is called assign:

From builder.md:

Vec3_t v;
Vec3_assign(&v, 1, 2, 3);
/* v does not have to be zero padded. */
Monster_pos_add(B, &v);

You can use that on child structs after you have a pointer to the struct. Assign takes care of zero padding.

[michbens]

I took ns_S1S2_field1_start() from the https://github.com/dvidelabs/flatcc/blob/master/doc/builder.md#structs

Vec3_t *v;
v = Monster_pos_start(B);
v->x = 1, v->y = 2, v->z = -3.2;
Monster_pos_end(B);

Anyway, my root is a table but I have many nested structs and I'm looking for a way to build the bin without breaking the all structure of the data.
Using assign doesn't help for that.

[mikkelfj]

Just keep in mind that nested structs in Flatbuffers work exactly like nedsted structs in C, when you are not using pointers.
As soon as you a pointer to the top-level struct, you can fill it in any way you like as a standard C struct, even using memcpy.

[mikkelfj]

Only, your own structs in C might not have the exact same layout as FlatBuffers due to alignment rules, but most C compilers will use the same alignment as FlatBuffers uses. A safer bet is to use the struct types generated by FlatBuffers in your own interfacing datastructures if possible.

[mikkelfj]

I think what you are looking for is:

static inline S1S2_t *S1S2_copy(S1S2_t *p, const S1S2_t *p2)
{ S1_copy(&p->field1, &p2->field1); S2_copy(&p->field2, &p2->field2);
  return p; }

static inline S1_t *S1_copy(S1_t *p, const S1_t *p2)
{ p->s1f1 = p2->s1f1; p->s1f2 = p2->s1f2;
  return p; }

static inline S2_t *S2_copy(S2_t *p, const S2_t *p2)
{ p->s2f1 = p2->s2f1; p->s2f2 = p2->s2f2;
  return p; }

If you want better performance you can look into from_pe versions, but that complicates matters, and it makes no difference on little endian platforms.

[mikkelfj]

I created a new branch assign_copy where _assign_copy is implemented.
https://github.com/dvidelabs/flatcc/tree/assign_copy

It does what you expected with the original create call, except you need to get the pointer first.
It also allows you to set a null pointer for any children, so you can have a shallow assignment.
This is entirely untested.

Se also builder.md under structs.
https://github.com/dvidelabs/flatcc/blob/assign_copy/doc/builder.md#structs

[michbens]

Yes this may be useful, thanks.

In builder.md, you wrote

S2_t s2;
S12_t s1s2;
S2_assign(&s2, 1, 2);
S1S2_assign_copy(&s1s2, 0, &s2, 3);
S1_assign(&s1s2.field1, 5, 6);

I guess you mean

S2_t s2;
S1S2_t s1s2;   // <------------------------------------ S1S2_t instead of S12_t
S2_assign(&s2, 1, 2);
S1S2_assign_copy(&s1s2, 0, &s2, 3);
S1_assign(&s1s2.field1, 5, 6);

[mikkelfj]

yeah thanks, if you find any more please let me know.

[michbens]

How can I use this branch ? is it on Master branch too ?

[mikkelfj]

For testing you can just clone the repo and check out the branch, then build it as usual.
As to master branch, I am awaiting some sort of feedback as to whether the feature is useful, works as intended, and other relevant feedback. If it is good, I'll merge to master.

[mikkelfj]

Tests added:

flatcc/test/monster_test/monster_test.c

Line 2050 in a0af9c1

int test_nested_struct(flatcc_builder_t *B)

[michbens]

Thanks for your help.
I'm trying this way:

ns_S1_t S1s = {1, 2};
ns_S2_t S2s = {3, 4};
ns_S1S2_t son1 = {S1s, S2s};

[mikkelfj]

Not to discurage you, but the portability of such constructs can be a bit shady. Clang issues a warning with clang -std=c89 -Wpedantic but not with c99 or c11. Furthermore, when used in temporary expressions such as function calls it can be more problematic, as opposed to const initializers.