[gatsby-source-contentful] Problem with JSON contentTypes

[TomiTakussaari]

So I tried Contentful plugin with our space, and this happens:

Fetching default locale
⡀ source and transform nodesdefault local is : en-US
⠄ source and transform nodescontentTypes fetched 4
Updated entries  8
Deleted entries  0
Updated assets  0
Deleted assets  0
Fetch Contentful data: 2666.023ms
success source and transform nodes — 2.864 s
error UNHANDLED REJECTION

  Error: Names must match /^[_a-zA-Z][_a-zA-Z0-9]*$/ but "200" does not.

  - Array.forEach

  - Array.forEach

  - Array.forEach

  - Array.forEach

npm ERR! code ELIFECYCLE

Our contentful space contains contentType that has Swagger api-docs in JSON fields, and Swagger api-docs has fields that are named like "200", or "400", which apparently cause this problem. (Confirmed by disabling JSON field from response)

"responses": {
    "200": {
        "schema": {
            "$ref": "#/definitions/PaymentDemandResponse"
        },
        "description": "successful operation"
    },
    "400": {
        "schema": {
            "$ref": "#/definitions/ErrorResponse"
        },
        "description": "Invalid input"
    }

I am not that familiar with Gatsby's GraphQL layer, or contentful plugin, but it seems that it tries to make JSON field part of the GraphQL schema (?), and numbers are obviously invalid field names there.

Should it just use JSON fields as plain strings instead ?

[TomiTakussaari]

So something like this:

          if (fieldType === `Object`) {
              entryItemFields[`${entryItemFieldKey}___NODE`] = createTextNode(entryNode, entryItemFieldKey, JSON.stringify(entryItemFields[entryItemFieldKey]), createNode);
              delete entryItemFields[entryItemFieldKey];
          }

next to this: https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby-source-contentful/src/normalize.js#L293

Consumers of this data would then have to use JSON.parse to actually use it as object.

Perhaps there is better way..

[KyleAMathews]

hmmm strange, this code here is supposed to fix this problem

gatsby/packages/gatsby-source-contentful/src/normalize.js

Line 31 in 6f56953

const fixId = id => {

[TomiTakussaari]

That only fixes fields named "id":

gatsby/packages/gatsby-source-contentful/src/normalize.js

Line 43 in 6f56953

deepMap(object, (v, k) => (k === `id` ? fixId(v) : v))

My problem seems to be that Contentful entry contains JSON field that contains fields that have number as a name.

[TomiTakussaari]

This reproduces the problem for me:

        {
            resolve: `gatsby-source-contentful`,
            options: {
                spaceId: `5mcft9q16uys`,
                accessToken: `e25be9e2b8266694d74e2709c63b436882fe9264240273c20a40dd1e4abdecbe`,
            },
        },

"gatsby": "1.9.70",
"gatsby-source-contentful": "^1.3.17",

[m4rrc0]

@TomiTakussaari I had a proposition similar to yours but instead of forcing JSON entries to be strings I added a "json" property to the object containing the whole object as string.
I guess in your case this would still be an issue because graphQL will still try to create the problematic keys.
Maybe your idea of importing objects as string is better. @Khaledgarbaya actualy agreed to that here #1703 and that is totally fine IMO too.

[m4rrc0]

Isn't this solved by PR #2967 ?

[Khaledgarbaya]

Isn't this solved by PR #2967?
That PR only adds a stringified version of the JSON to the node.

Tha problem happens when you have JSON with numbers as keys or anything that does not pass /^[_a-zA-Z][_a-zA-Z0-9]*$/ test, this a GraphQL requirement.
Being able to query the JSON object using GraphQL is nice but I think it won't work for all case because we can't control what you a user writes there.

I can see few options to solve this.

• the source plugin detect these invalid fields and prepend something, which I am not a big fan of altering the data
• make the JSON queryable option optional, you will have always the stringified version always.
• Make the JSON only available as a string.

I'll be happy to create a Proposal issue as this might be a breaking change depends on the option.

@KyleAMathews @MarcCoet @TomiTakussaari @sarahatwork

[TomiTakussaari]

For me, making JSON field only available as a string makes most sense.

Having "JSON queryable" option would mean that if option is enabled, its possible to break site deploy by just adding wrong kind of JSON to any JSON field in Contentful space.
So if option approach is chosen, it should IMO be a whitelist of ContentTypes.

Also I can't even think of a use case where I would like have my JSON field data queryable when using Contentful, I feel like its not proper way to use Contentful.
JSON field data should just be data to be used somewhere. There could be other opinions though :)

[Khaledgarbaya]

I am also for keeping just the text content of the JSON.

[KyleAMathews]

Yeah, that would simplify things if people only could query the JSON string which they JSON.parse client side.

[m4rrc0]

Alright. I see. JSON as string is perfect for me too. I only use it like that anyway.

[Khaledgarbaya]

ok cool I am a bit under the weather these days, once I am feeling better i will create a PR, should be in the next week.

[KyleAMathews]

Due to the high volume of issues, we're closing out older ones without recent activity. Please open a new issue if you need help!

[m4rrc0]

Apparently this still hasn't been implemented...
JSON nodes still get populated with what is inside the JSON field.

Surely this must be easy to work around with the new schema customization API but I get a node validation error that won't go away. I tried a few things similar to:

exports.createSchemaCustomization = ({ actions, schema }) => {
  const { createTypes } = actions
  const typeDefs = [
    `
    type contentfulBlablaJsonNode implements Node @dontInfer {
      id: ID!
    }
    `,
  ]
  createTypes(typeDefs)
}

I guess validation happens before the createSchemaCustomization hook. Is there a way to modify the node before it gets validated?