Unicode characters in pipeline names break buildkite-metrics when sending to CloudWatch

[rbvigilante]

We've been setting up buildkite-metrics at Redbubble. One of our pipelines happened to have an emoji in its name; Unicode support is awesome like that.

Unfortunately, CloudWatch is not quite so awesome, and it throws an error when buildkite-metrics tries to push metrics under a namespace with non-ASCII characters in it. This causes buildkite-metrics to crash, like this:

...
2017/11/16 09:46:16 Submitting chunk of 10 metrics to Cloudwatch
2017/11/16 09:46:16 Submitting chunk of 10 metrics to Cloudwatch
2017/11/16 09:46:16 Submitting chunk of 10 metrics to Cloudwatch
2017/11/16 09:46:17 Submitting chunk of 10 metrics to Cloudwatch
InvalidParameterValue: The value for parameter MetricData.member.2.Dimensions.member.1.Value contains non-ASCII characters.

I would expect the same problem to exist for queues, but I haven't tested this. I also don't know how StatsD copes with Unicode.

I started trying to put together a patch for this, but I'm not sure what the desired behaviour in this case is. I can think of four potential options:

1. 'Officially' support only ASCII names in buildkite-metrics; check all the pipeline and queue names before attempting to send metrics and exit with a better error message if any Unicode characters are found. This leaves a bit of a bitter taste in my mouth, though; it seems like a metrics system for a product that officially supports Unicode shouldn't break down when Unicode is used.
2. Just don't post metrics for pipelines with Unicode names. This also feels a bit dirty, but is possibly a reasonable first step in that it would at least get metrics out for the other pipelines. It would also need to be transparent about doing this somehow; people might get quite confused by this behaviour.
3. Publish the metrics, removing the non-ASCII characters. There's a potential name-collision issue here, and some names might end up truncated into strange things.
4. Publish the metrics, replacing the Unicode characters with some ASCII character(s). The ideal here would be to be intelligent about which character to use (i.e, "é" to "e" and "ü" to "u"), but doing this appears to be quite complex, so it might be more feasible to just replace all of them with some uncommon ASCII character (asterisk or something). This also has a potential name collision issue.

I'd appreciate any opinions on this.

(We can, and probably should, also ask Amazon very nicely to add Unicode support to CloudWatch, but I have a feeling that even if they'll do it, we won't see it soon)

[lox]

Interesting, I think we have some special handling of this problem for github statuses (similar problem). I'd say we should punycode encode (https://godoc.org/golang.org/x/net/idna) the pipeline names.

Thanks for the report and the careful thought around suggestions!

[rbvigilante]

That's a heaps better idea. Awesome.

[toolmantim]

This is fixed and released in 2.0.0 @rbvigilante!

[rbvigilante]

Awesome. Thanks for looking at this so quickly!