Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Help with unions of complex types? #239

Closed
haysmike opened this issue Jun 14, 2019 · 3 comments
Closed

Help with unions of complex types? #239

haysmike opened this issue Jun 14, 2019 · 3 comments

Comments

@haysmike
Copy link

haysmike commented Jun 14, 2019

Hello! I'm trying to implement a union type and running into an error. It seems like others are having some success (#174) so I'm probably just missing something, but any help would be much appreciated.

Code to reproduce the issue:

const avro = require('avsc');

const type = avro.Type.forSchema({
  type: 'record',
  fields: [{
    name: 'blah',
    type: [
      {type: 'record', name: 'union1', fields: [{name: 'foo', type: 'string'}]},
      {type: 'record', name: 'union2', fields: [{name: 'bar', type: 'int'}]}
    ]
  }]
});

const buf = type.toBuffer({blah: {foo: 'baz'}});
const val = type.fromBuffer(buf);
console.log(val)

Error:

/Users/murkey/code/avro-test/node_modules/avsc/lib/types.js:3017
  throw new Error(f('invalid %j: %j', type.schema(), val));
  ^

Error: invalid [{"name":"union1","type":"record","fields":[{"name":"foo","type":"string"}]},{"name":"union2","type":"record","fields":[{"name":"bar","type":"int"}]}]: {"foo":"baz"}
    at throwInvalidError (/Users/murkey/code/avro-test/node_modules/avsc/lib/types.js:3017:9)
    at WrappedUnionType._write (/Users/murkey/code/avro-test/node_modules/avsc/lib/types.js:1496:7)
    at RecordType.writeRecord$ [as _write] (eval at RecordType._createWriter (/Users/murkey/code/avro-test/node_modules/avsc/lib/types.js:2312:10), <anonymous>:4:6)
    at RecordType.Type.toBuffer (/Users/murkey/code/avro-test/node_modules/avsc/lib/types.js:646:8)
    at Object.<anonymous> (/Users/murkey/code/avro-test/index.js:14:18)
    at Module._compile (module.js:653:30)
    at Object.Module._extensions..js (module.js:664:10)
    at Module.load (module.js:566:32)
    at tryModuleLoad (module.js:506:12)
    at Function.Module._load (module.js:498:3)

I've also tried wrapping the union schemas with avro.Type.forSchema:

    type: [
      avro.Type.forSchema({type: 'record', name: 'union1', fields: [{name: 'foo', type: 'string'}]}),
      avro.Type.forSchema({type: 'record', name: 'union2', fields: [{name: 'bar', type: 'int'}]})
    ]

But this produced the same error.

Any ideas? Thanks :)

@haysmike
Copy link
Author

haysmike commented Jun 14, 2019

Oh, interesting. It seems like it works if I do:

const buf = type.toBuffer({blah: {union1: {foo: 'baz'}}});

Unfortunately this conflicts with the types generated by https://github.com/joewood/avro-typescript. Maybe a bug on their end?

@mtth
Copy link
Owner

mtth commented Jun 14, 2019

You're correct, by default you'll need to "wrap" the union's values so that no information is lost (more information here). I'm not familiar with avro-typescript but I suspect the library is currently generating types for unwrapped unions. Perhaps there is a way to make it generate the wrapped equivalent?

Depending on the structure of your data, you might also be able to use logical types to resolve the ambiguity and avoid wrapping the types. This comment includes an example for how to do so, as well as a little more context.

@haysmike
Copy link
Author

Appreciate the quick response! That makes sense - I'll read up on those links and ask on the avro-typescript repo if I'm still stuck. Thanks again :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants