Timestamp precision

[13ogdan]

In our project, we have an issue with searing for fast operations. If the difference between the two events is less than 1 millisecond _time property cannot be used in proper way, as both event has the same timestamp.
The solution is pretty simple and didn't find any drawbacks, change in epoch digits number from 3 to 6:

    internal static class EpochExtensions
    {
        private static DateTimeOffset Epoch = new DateTimeOffset(1970,1,1,0,0,0,TimeSpan.Zero);

        public static double ToEpoch(this DateTimeOffset value)
        {
            // From Splunk HTTP Collector Protocol
            // The default time format is epoch time format, in the format <sec>.<ms>. 
            // For example, 1433188255.500 indicates 1433188255 seconds and 500 milliseconds after epoch, 
            // or Monday, June 1, 2015, at 7:50:55 PM GMT.
            // See: http://dev.splunk.com/view/SP-CAAAE6P

            return Math.Round((value - Epoch).TotalSeconds, 6, MidpointRounding.AwayFromZero);
        }
    }

[VictorioBerra]

@13ogdan Can you answer this? #172 (comment)

[hangy]

The official docs (web archive) only have examples with three decimals, but don't appear to limit it to three decimals. I've read that a better precision with more decimals is possible in the past.

[VictorioBerra]

CC @hangy @13ogdan @EEParker

I am working to test this now, I created a Splunk docker container via:

docker run -d -p 8000:8000 -p 8088:8088 -e SPLUNK_START_ARGS='--accept-license' -e SPLUNK_PASSWORD='<password>' splunk/splunk:latest

I logged into the Web UI, and I created a HEC token, I also disabled SSL.

I checked out the PR via gh pr checkout 172

I ran the sample, and I toggled 3,6,9 via the new SubSecondPrecision enum and submited an event for each.

The UI only seems to support 3:

I fond the following thread: https://community.splunk.com/t5/Getting-Data-In/Time-parsing-working-correctly-but-not-able-to-see-nanoseconds/td-p/677657

It appears that I need to define a parsing rule to handle the 6/9 options. Is there a way we can confirm this works? I think we need to at least see some proof Slack is accepting the precision before we merge that PR.

[EEParker]

I was able to confirm "Microseconds" (6) work with a props.conf file, but I couldn't get parsing rules to evaluate nanoseconds (9). There might be more config needed on the Splunk side.

* | sort - _time | eval time=strftime(_time, "%Y-%m-%d %H:%M:%S:%9N") | table _time, time, Level, RenderedMessage