AtlasMap mapping definition not bound to the mapping step

[lburgazzoli]

While testing camel-k and atlasmap I've noticed that clicking to the done button after having defined the mapping seems to do noting but after a while the mapping editor is closed.

When the integration is the deployed, the mapping step lacks mapping definition.

[riccardo-forina]

I tried to reproduce this locally. I updated the backend - on branch 1.6.x btw - and run the UI in dev mode. I can create an integration as usual, with the mapping and all. The payload looks correct.
So I can't reproduce this really.

There is only one thing that looks off, and it's this error when entering the configuration page of any connector action:

[gashcrumb]

Yeah, that bad request is new. The UI hits that endpoint to fetch data shapes and the config properties object.

Actually nowadays it also hits that endpoint whenever steps are added and removed. Error responses like this just get ignored though when the request is made in this fashion.

[lburgazzoli]

@gashcrumb so I did another test deleting everything (minishift, browser caches) and now I do see the same errors as @riccardo-forina

but I got the mapping step properly configured but now it is @nicolaferraro that has the same issue :)

[zregvart]

That 400 (Bad Request) must leave some trace of an error somewhere, either in the meta pod or the server pod or even in the response itself. When I tried to reproduce this I focused on trying to make the call to meta pod fail, and in that case I do get 400 (Bad Request) but with _meta.message and _meta.type which in turn end up being handled in the UI and present a warning message. So no way to proceed with action editing.

[gashcrumb]

Had you maybe not rebuilt your UI image in awhile or something perhaps?

I think the bad request is a red herring, it's getting made as the component initializes. Looks like the UI isn't sending an empty object or anything at this point, so could be causing the bad request. But then a second request gets made using an empty object, that succeeds alright and then it's fine from there. I'll see about tracking down that initial request though.

[nicolaferraro]

This is really strange.. I hit this problem now, but this morning it was working.. and no redeployments in the meantime.

[zregvart]

@lburgazzoli @nicolaferraro seems that others are having difficult time reproducing this, can you write down the steps taken that manifest this issue?

@syndesisio/qe anyone of you experienced something similar to what's described here?

[asmigala]

@zregvart one way to semi-reliably reproduce this is to create an API Provider integration from the todo API, implement Fetch Task and just map the input id to the output id, nothing else. You might have to build it a couple of times to get it though.

EDIT: not sure this is right, I wrote this based on our chat conversation, but now I'm thinking this might be a different issue, my problem happens when generating the integration tar to hand off to s2i.

[zregvart]

@asmigala here's the bit from our chat that reminded me of this issue, the exception you got when you tried to publish the integration.

2019-03-15 11:03:12.559  INFO [-,,,] 1 --- [tion Controller] i.s.s.c.i.online.PublishHandler          : Integration [todo-integration]: Created project files and starting build
Exception in thread "pool-11-thread-1" java.lang.IllegalStateException: Missing configured property for data mapper mapping definition
	at io.syndesis.integration.project.generator.ProjectGenerator.addMappingRules(ProjectGenerator.java:347)
	at io.syndesis.integration.project.generator.ProjectGenerator.lambda$generateAddProjectTarEntries$5(ProjectGenerator.java:289)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

It alludes to a missing mapping that @lburgazzoli also noticed.
That made me think that the root cause might be the same. And if we fix the root cause: why is the mapping missing? -- we might fix both issues.

[asmigala]

If we add a 2s delay in our automated tests between selecting a mapping and clicking the Done button, it seems the issue does not happen. So that might be a reason why it's tricky to reproduce locally.

[heiko-braun]

@asmigala Based on your last comment in #4840 (comment) I would say we can lower the priority to p1. What do you think?

[asmigala]

@heiko-braun not sure, I assume @lburgazzoli originally found it manually. The problem is that if it does happen, the whole integration is completely broken.

[zregvart]

I might be hallucinating, so if it doesn't make sense for any reason please disregard this.

So I think this might have to do with #4914 where we might change the data shape and the change in the data shape might trigger a change in the mapping configuration.

Not sure @gashcrumb do you think this might be the case, or am I totally of the rails?

[gashcrumb]

@zregvart the mapping definition is only set on the step here where mappings comes from the data mapper via it's save handler here. Which means you have to actually visit the data mapper page to cause changes to happen in that fashion.

You know unless there's another bug that we've yet to uncover, but yeah, trouble is I've not been able to reproduce so it's hard to say.

I'd say that because the way the mapping is bubbled up from the data mapper host component it could be possible to perhaps very quickly do a mapping and then click 'Done' and wind up with an unset atlasmap config property, but I wasn't able to make that happen.

[zregvart]

I guess @asmigala tell us how to run just that test from syndesis-qe, I think I was able to launch tests from the IDE.

[gashcrumb]

Ah this comment is probably it. There's a bit of asynchronous communication between the data mapper and the parent controller, so if you quickly press done after creating those mappings I can see how this can happen. Would require some rework to fix properly but I can see if I can figure out another solution.

[gashcrumb]

So have done some testing here. I actually reproduced this on the first integration I created after a fresh new Syndesis install. Made some code changes, I couldn't reproduce it after that, backed out said changes, still couldn't reproduce it. So now I've been bouncing backend stuff to see if I can get it to reproduce reliably or a second time on the same install, no luck there yet.

I will say that normally the callback we supply to the data mapper component fires almost immediately when you release the mouse button on the last part of creating a mapping. The only time it doesn't appear to be so quickly is if I go and redeploy the syndesis-server pod, then there's a startup lag to the data mapper and the first requests out to the data mapper endpoint seem to take a bit longer. This appears to have an impact on when the save callback handler fires which is where we then propagate the saved mapping to parent controller. If that callback handler doesn't fire by the time you click done we've no mapping to save.

[gashcrumb]

Can we ensure the data mapper-related backend bits are initialized when the server starts up vs. on demand? That'd probably help for this.

[gashcrumb]

While we ponder that, I'll ensure the "Done" button is disabled while the component hasn't received a mapping, which will at least prevent the user from moving on and creating a mapping step without mapping data.

[mmelko]

I wasn't able to reproduce this on CR1 (1.6.4-fuse...). Closing.