fix: do not throw error if VALUE_DELIMITER is set on non-DELIMITED topic

[agavra]

fixes #4200

Description

The VALUE_DELIMITER only makes sense in the context of DELIMITED format and today we throw an error if it is set in any other format. This PR removes that error and just ignores it if it is set on a non-delimited stream. This way:

1. users can issue commands that take a delimited topic and turn it into another format
2. if they go back to delimited (e.g. delimited->json->delimited) it will maintain the original source delimiter

Alternatively, we can just not pipe the delimiter and use the default if going from a non-delimited format back to delimited.

Testing done

• updated unit tests
• updated qtt
• local end-to-end testing

Reviewer checklist

• Ensure docs are updated if necessary. (eg. if a user visible feature is being added or changed).
• Ensure relevant issues are linked (description should include text like "Fixes #")

[stevenpyzhang]

extra new line?

[agavra]

it's intentional - I thought it was cleaner to read with a new line than without

[stevenpyzhang]

lgtm

[big-andy-coates]

Would a C* statement with DELIMITED set for a none DELIMIITED format no longer fail with this change? e.g.

CREATE TABLE FOO (...) WITH (VALUE_FORMAT='JSON', DELIMITER=',');

???

Allowing the above would be poor UX IMHO, as we're allowing the user to set something that doesn't make sense. If this no longer fails then I think we need a follow up PR to check for DELIMITER purely in CT/CS statements.

[agavra]

This is correct, I had thought about that (see the description) - it no longer fails, and I think that's OK. It does do something, it indicates that any DELIMITED format down the line will use this delimiter. The alternative would be to have the delimiter change when going between formats and back to delimited. Thoughts?