[Relay][Frontend][Keras] NHWC import support.

[jwfromm]

Currently the Keras importer can only output models with NCHW layout. This PR adds a layout argument to the Keras importer that allows NHWC to be used instead. This is useful for platforms that may have better NHWC than NCHW performance such as the Raspberry Pi.

A few notes:

• Xception is not being tested in NHWC mode as I found that it was causing a strange cuda error when targeting GPU. On CPU, it works fine however so I suspect this is an issue with the nhwc separable convolutions in Topi rather than the importer.
• I made a few changes to the test script to allow eager execution (and TF2.0) to work.

[jwfromm]

@masahi, @soiferj Can you take a look at this PR?

[masahi]

I'm not familiar with keras, but I wonder why keras use NCHW while tensorflow uses NHWC? (I'm not familiar with tf either). I heard that since TF 2.0, keras became their official frontend or something like that.

[jwfromm]

@masahi, Keras does use NHWC by default just like TensorFlow. However, because TVM is better optimized for NCHW, the Keras frontend previously converted all layers from NHWC to NCHW. This option allows us to keep the layers in their original format.

[jwfromm]

@comaniac I think I've fixed all the style issues you pointed out.

[comaniac]

LGTM. Thanks :)

[anijain2305]

I would suggest to use ConvertLayout. ConvertLayout was designed for exactly this purpose.

[jwfromm]

@anijain2305, I think you're right that for most cases ConvertLayout will achieve something similar. However, there are some benefits to having layers directly parsed as NHWC. I actually did this PR because I'm working on a project that uses some funky custom Keras layers and it's very useful to parse them directly in NHWC.

[tqchen]

Given that the model is in native NHWC, perhaps it is better to do it in another way around(note that the old approach converts to NCHW on the fly), ingest as NHWC as in this PR and then call ConvertLayout

[jwfromm]

@tqchen, you're right that would be a more direct way to apply ConvertLayout. If everyone else would prefer it, I could change the Keras frontend to only output in NHWC and then we would rely on ConvertLayout to get models into NCHW.

[anijain2305]

I see. Yes, I meant that we should retain native framework format (NHWC in this case). And use ConvertLayout to change the layout to desired layout (passed on by relay.from_keras API).

@jwfromm Yes, in my opinion, that is the cleanest way forward. Same thing needs to happen for TF framework. I can take care of that. Will appreciate if you can take care of Keras.

[tqchen]

That sounds good.

[tqchen]

Merging this for now as this PR is self contained. It would be great if we can have a followup PR that updates the frontend to use ConvertLayout from NHWC -> NCHW so we won't have to support two path in the frontend

Thanks @anijain2305 @jwfromm @comaniac