Allow multiple index-patterns per index-pattern

[yehosef]

The title sounds confusing - but the idea is that sometimes I have and index pattern which has multiple timestamps and I would like to be able to use it in multiple ways. Eg, I have a create_at field and a modified_at field. The default timestamp of the index is "created_at" but I wanted to do a chart where I do a terms aggregation on things modified recent. While I can create a date histogram using a different field, the time selector at the top of the page filters based on the primary timestamp field.

If I would want to create a dashboard with current week's events and I have one chart which show new users and signup plans and another with recent member up/downgrades, I'll have a problem.

It would be better if we could specify different "index-sets" (I made that up..) that use the same "index-patterns". This would also help me to solve a different problem I sometimes have where I have bad data which is missing the time field - currently that data is completely lost but it I could create an "index-set" that doesn't include a timefield, I could then create a visualization widget that would help me see that.

[ThomasFlanaghan]

Hi,

Can you not just use aliases for this? We use multiple aliases on the same index to create different index patterns for that one index.
https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-aliases.html

[yehosef]

It's a work around - but it seems annoying/wrong that I need to make a change in the actual datastore in order to accommodate a design decision in the visualization tool. If I have many patterns with this approach, I think it becomes cumbersome, especially since there are no native tools in kibana for creating an alias like there is in kopf/cerebro (console doesn't count, people making dashboards should have to use the "console" to see a data source in different ways.)

Because the actual index pattern is currently used as the reference it is hard/not possible now. But if there was just an "reference" or "name" field for the index pattern it would be pretty simple to accomplish this. This field could default to the index pattern allowing you to override it if you want to make a new alias for the pattern. There are other approaches, but none is very complicated and I think would be very useful.

[epixa]

This use case definitely sounds reasonable, though I'm not sure if the "index-sets" approach is necessarily the right one in the long term.

One possible alternative option might be a configuration you can specify on each individual embedded object in a dashboard to override behaviors such as which time field to filter on for that specific visualization.

There's a lot of hidden complexity here one way or another. For example, while these options would solve this specific use case, they make it a bit awkward to deal with different time fields in an index pattern in other apps, such as discover. If items on a dashboard can filter based on different time fields within the same indices, why can't we do the same thing in discover without switching to an entirely different index pattern?

[yehosef]

The index-sets name might be misleading and is probably bad. This is what I mean - there are two ways I can think of to solve this. One is to create another top level container that is the same level as the index pattern WRT the dashboards, etc. The idea is that you would call it "index name" which could be anything - just unique - and it would have an index-pattern and timefield (or not). These would be independent from each other even though they are using the same index-pattern (underlying data). Meaning that they could have their own scripted fields, source filters, field value settings, etc. The second option is to have the scripted field, source filters, etc be shared and used for an index pattern and then have some other variable that would be called "index name" with the timefield to use (or not). This could be set to default to the index pattern if not given. I think the first is more powerful and seems easier to implement - but think the second is also fine and has some advantages if I use it a lot and want the settings I use for field filters, scripted field, etc to be shared (which is reasonable).

…

On Wed, Aug 23, 2017 at 5:18 PM, Court Ewing ***@***.***> wrote: This use case definitely sounds reasonable, though I'm not sure if the "index-sets" approach is necessarily the right one in the long term. One possible alternative option might be a configuration you can specify on each individual embedded object in a dashboard to override behaviors such as which time field to filter on for that specific visualization. There's a lot of hidden complexity here one way or another. For example, while these options would solve this specific use case, they make it a bit awkward to deal with different time fields in an index pattern in other apps, such as discover. If items on a dashboard can filter based on different time fields within the same indices, why can't we do the same thing in discover without switching to an entirely different index pattern? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#13605 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAJHBWm6qwwFBT8h-JhWNbXtByvPcLfkks5sbDTRgaJpZM4O8kvC> .

[webmstr]

Jump in the wayback machine and join me in 2015 for issue 3351.

[yehosef]

@webmstr - seems like this is just a duplicate then - or at least linked.

[quickdraw6906]

@webmstr I agree linked. Not a duplicate.

[chrisronline]

#3351 encapsulates this already so closing this one.