Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider the case when log stream has no log events #18

Merged
merged 1 commit into from
Apr 6, 2020
Merged

Consider the case when log stream has no log events #18

merged 1 commit into from
Apr 6, 2020

Conversation

yutachaos
Copy link
Collaborator

@yutachaos yutachaos commented Mar 24, 2020

Issue #, if available:

Description of changes:

  • If the log stream has no log events, describe-log-stream output does not include LastIngestionTime, so it becomes panic due to nil pointer dereference。

Reproduce

# NG pattern
aws logs create-log-group --log-group-name test-log-group
aws logs create-log-stream --log-group-name test-log-group --log-stream-name test-log-stream
aws logs describe-log-streams --log-group-name test-log-group --log-stream-name test-log-stream
{
    "logStreams": [
        {
            "logStreamName": "test-log-stream",
            "creationTime": 1585056526451,
            "arn": "arn:aws:logs:ap-northeast-1::log-group:test-log-group:log-stream:test-log-stream",
            "storedBytes": 0
        }
    ]
}
aws logs get-log-events --log-group-name test-log-group --log-stream-name test-log-stream
{
    "events": [],
    "nextForwardToken": "hogehoge",
    "nextBackwardToken": fugafuga"
}
utern test-log-group
+ test-log-group
⠋ Fetching log streams... panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x14f82d6]

goroutine 1 [running]:
github.com/knqyf263/utern/cloudwatch.(*Client).ListStreams.func1(0xc000332cc0, 0xc0004c4c01, 0x10000c0004073e0)
	/home/travis/gopath/src/github.com/knqyf263/utern/cloudwatch/stream.go:32 +0xf6
github.com/aws/aws-sdk-go/service/cloudwatchlogs.(*CloudWatchLogs).DescribeLogStreamsPagesWithContext(0xc00000e040, 0x17b4da0, 0xc000089b40, 0xc0004896b0, 0xc0002cd4a8, 0x0, 0x0, 0x0, 0xc0002cd4c8, 0x14853f4)
	/home/travis/gopath/pkg/mod/github.com/aws/[email protected]/service/cloudwatchlogs/api.go:1666 +0x163
github.com/knqyf263/utern/cloudwatch.(*Client).ListStreams(0xc0000680b0, 0x17b4da0, 0xc000089b40, 0xc000374ff0, 0xe, 0x1710cb72770, 0x1b48838, 0x0, 0x0, 0x0, ...)
	/home/travis/gopath/src/github.com/knqyf263/utern/cloudwatch/stream.go:66 +0x1d2
github.com/knqyf263/utern/cloudwatch.(*Client).Tail(0xc0000680b0, 0x17b4da0, 0xc000089b40, 0xc000089b40, 0xc000200580)
	/home/travis/gopath/src/github.com/knqyf263/utern/cloudwatch/cloudwatch.go:100 +0x782
main.run(0xc0001f4e70, 0xc0001f4e70, 0x0)
	/home/travis/gopath/src/github.com/knqyf263/utern/main.go:121 +0x8f
main.main.func1(0xc0000d86e0, 0x0, 0x0)
	/home/travis/gopath/src/github.com/knqyf263/utern/main.go:90 +0x8d
github.com/urfave/cli.HandleAction(0x1557380, 0x1631ed8, 0xc0000d86e0, 0x0, 0x0)
	/home/travis/gopath/pkg/mod/github.com/urfave/[email protected]/app.go:523 +0xbe
github.com/urfave/cli.(*App).Run(0xc0001fe000, 0xc0000aa040, 0x2, 0x2, 0x0, 0x0)
	/home/travis/gopath/pkg/mod/github.com/urfave/[email protected]/app.go:285 +0x5df
main.main()
	/home/travis/gopath/src/github.com/knqyf263/utern/main.go:93 +0x907

# OK pattern

TIMESTAMP=$(node -e 'console.log(Date.now())')
aws logs put-log-events --log-group-name test-log-group --log-stream-name test-log-stream --log-events timestamp=$TIMESTAMP,message='test'
aws logs describe-log-streams --log-group-name test-log-group --log-stream-name test-log-stream
utern test-log-group
aws logs put-log-events --log-gm --log-events timestamp=$TIMESTAMP,message='test'g-strea
{
    "nextSequenceToken": "fuga"
}
aws logs describe-log-streams --log-group-name test-log-group --log-stream-name test-log-stream
{
    "logStreams": [
        {
            "logStreamName": "test-log-stream",
            "creationTime": 1585056526451,
            "firstEventTimestamp": 1585056572981,
            "lastEventTimestamp": 1585056572981,
            "lastIngestionTime": 1585056573954,
            "uploadSequenceToken": "mogamoga",
            "arn": "arn:aws:logs:ap-northeast-1:xxxxxxx:log-group:test-log-group:log-stream:test-log-stream",
            "storedBytes": 0
        }
    ]
}

utern test-log-group
+ test-log-group
+ test-log-group › test-log-stream (2020-03-24T22:29:33+09:00)
test-log-group test-log-stream test

@knqyf263
Copy link
Owner

knqyf263 commented Mar 26, 2020

Thank you for fixing the bug! I'd like to make sure uploadSequenceToken is always nil if a log stream is empty and the opposite as well. Also, I'm wondering if lastIngestionTime is always not nil with non-nil uploadSequenceToken.

@yutachaos
Copy link
Collaborator Author

yutachaos commented Apr 6, 2020

Thank you for fixing the bug! I'd like to make sure uploadSequenceToken is always nil if a log stream is empty and the opposite as well. Also, I'm wondering if lastIngestionTime is always not nil with non-nil uploadSequenceToken.

Thank for your comment.
If there is no log event in the log stream, FirstEventTimestamp, LastEventTimestamp, LastIngestionTime, and UploadSequenceToken will be nil.
This activity is not officially documented.
However, it is confirmed by a support ticket on AWS.

Added FirstEventTimestamp, LastEventTimestamp, LastIngestionTime, and UploadSequenceToken nil check.

@knqyf263 knqyf263 merged commit f630140 into knqyf263:master Apr 6, 2020
@knqyf263
Copy link
Owner

knqyf263 commented Apr 6, 2020

Thanks a lot!

@yutachaos yutachaos deleted the feature/fixed_log_stream_has_no_log_events branch April 9, 2020 08:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants