Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: How to distinguish whether the 5xx status code (eg 500) comes from upstream or apisix #2817

Merged
merged 16 commits into from
Nov 27, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions apisix/init.lua
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ local type = type
local ngx_now = ngx.now
local str_byte = string.byte
local str_sub = string.sub
local tonumber = tonumber
local load_balancer
local local_conf
local dns_resolver
Expand Down Expand Up @@ -626,9 +627,28 @@ local function common_phase(phase_name)
end


local function set_resp_upstream_status(up_status)
core.response.set_header("X-APISIX-Upstream-Status", up_status)
core.log.info("X-APISIX-Upstream-Status: ", up_status)
end


function _M.http_header_filter_phase()
core.response.set_header("Server", ver_header)

local up_status = get_var("upstream_status")
if up_status and #up_status == 3
and tonumber(up_status) >= 500
and tonumber(up_status) <= 599
then
set_resp_upstream_status(up_status)
elseif up_status and #up_status > 3 then
local last_status = str_sub(up_status, -3)
if tonumber(last_status) >= 500 and tonumber(last_status) <= 599 then
set_resp_upstream_status(up_status)
end
end

common_phase("header_filter")
end

Expand Down
1 change: 1 addition & 0 deletions doc/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@
* [Changelog](../CHANGELOG.md)
* [Benchmark](benchmark.md)
* [Code Style](../CODE_STYLE.md)
* [Debug Function](./debug-function.md)
* [FAQ](../FAQ.md)

## Plugins
Expand Down
151 changes: 151 additions & 0 deletions doc/debug-function.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
<!--
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
-->

[Chinese](zh-cn/debug-function.md)

## `5xx` response status code

Similar `5xx` status codes such as 500, 502, 503, etc., are the status codes in response to a server error. When a request has a `5xx` status code; it may come from `APISIX` or `Upstream`. How to identify the source of these response status codes is a very meaningful thing. It can quickly help us determine the problem.

## How to identify the source of the `5xx` response status code

In the response header of the request, through the response header of `X-APISIX-Upstream-Status`, we can effectively identify the source of the `5xx` status code. When the `5xx` status code comes from `Upstream`, the response header `X-APISIX-Upstream-Status` can be seen in the response header, and the value of this response header is the response status code. When the `5xx` status code is derived from `APISIX`, there is no response header information of `X-APISIX-Upstream-Status` in the response header. That is, only when the status code of `5xx` is derived from Upstream will the `X-APISIX-Upstream-Status` response header appear.

## Example

>Example 1: `502` response status code comes from `Upstream` (IP address is not available)

```shell
$ curl http://127.0.0.1:9080/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
"methods": ["GET"],
"upstream": {
"nodes": {
"127.0.0.1:1": 1
},
"type": "roundrobin"
},
"uri": "/hello"
}'
```

Test:

```shell
$ curl http://127.0.0.1:9080/hello -v
......
< HTTP/1.1 502 Bad Gateway
< Date: Wed, 25 Nov 2020 14:40:22 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 154
< Connection: keep-alive
< Server: APISIX/2.0
< X-APISIX-Upstream-Status: 502
<
<html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>openresty</center>
</body>
</html>

```

It has a response header of `X-APISIX-Upstream-Status: 502`.

>Example 2: `502` response status code comes from `APISIX`

```shell
$ curl http://127.0.0.1:9080/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
"plugins": {
"fault-injection": {
"abort": {
"http_status": 500,
"body": "Fault Injection!\n"
}
}
},
"uri": "/hello"
}'
```

Test:

```shell
$ curl http://127.0.0.1:9080/hello -v
......
< HTTP/1.1 500 Internal Server Error
< Date: Wed, 25 Nov 2020 14:50:20 GMT
< Content-Type: text/plain; charset=utf-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< Server: APISIX/2.0
<
Fault Injection!
```

There is no response header for `X-APISIX-Upstream-Status`.

>Example 3: `Upstream` has multiple nodes, and all nodes are unavailable

```shell
$ curl http://127.0.0.1:9080/apisix/admin/upstreams/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
"nodes": {
"127.0.0.3:1": 1,
"127.0.0.2:1": 1,
"127.0.0.1:1": 1
},
"retries": 2,
"type": "roundrobin"
}'
```

```shell
$ curl http://127.0.0.1:9080/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
"uri": "/hello",
"upstream_id": "1"
}'
```

Test:

```shell
$ curl http://127.0.0.1:9080/hello -v
< HTTP/1.1 502 Bad Gateway
< Date: Wed, 25 Nov 2020 15:07:34 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 154
< Connection: keep-alive
< Server: APISIX/2.0
< X-APISIX-Upstream-Status: 502, 502, 502
<
<html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>openresty</center>
</body>
</html>
```

It has a response header of `X-APISIX-Upstream-Status: 502, 502, 502`.
1 change: 1 addition & 0 deletions doc/zh-cn/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
* [变更日志](../../CHANGELOG_CN.md)
* [压力测试](benchmark.md)
* [代码风格](../../CODE_STYLE_CN.md)
* [调试功能](./debug-function.md)
* [常见问答](../../FAQ_CN.md)

## 插件
Expand Down
151 changes: 151 additions & 0 deletions doc/zh-cn/debug-function.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
<!--
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
-->

[English](../debug-function.md)

## `5xx` 响应状态码

500、502、503等类似的 `5xx` 状态码,是由于服务器错误而响应的状态码,当一个请求出现 `5xx` 状态码时;它可能来源于 `APISIX``Upstream` 。如何识别这些响应状态码的来源,是一件很有意义的事,它能够快速的帮助我们确定问题的所在。

## 如何识别 `5xx` 响应状态码的来源

在请求的响应头中,通过 `X-APISIX-Upstream-Status` 这个响应头,我们可以有效的识别 `5xx` 状态码的来源。当 `5xx` 状态码来源于 `Upstream` 时,在响应头中可以看到 `X-APISIX-Upstream-Status` 这个响应头,并且这个响应头的值为响应的状态码。当 `5xx` 状态码来源于 `APISIX` 时,响应头中没有 `X-APISIX-Upstream-Status` 的响应头信息。也就是只有 `5xx` 状态码来源于 `Upstream` 时,才会有 `X-APISIX-Upstream-Status` 响应头。

## 示例

示例1:`502` 响应状态码来源于 `Upstream` (IP地址不可用)

```shell
$ curl http://127.0.0.1:9080/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
"methods": ["GET"],
"upstream": {
"nodes": {
"127.0.0.1:1": 1
},
"type": "roundrobin"
},
"uri": "/hello"
}'
```

测试:

```shell
$ curl http://127.0.0.1:9080/hello -v
......
< HTTP/1.1 502 Bad Gateway
< Date: Wed, 25 Nov 2020 14:40:22 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 154
< Connection: keep-alive
< Server: APISIX/2.0
< X-APISIX-Upstream-Status: 502
<
<html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>openresty</center>
</body>
</html>

```

具有 `X-APISIX-Upstream-Status: 502` 的响应头。

示例2: `502` 响应状态码来源于 `APISIX`

```shell
$ curl http://127.0.0.1:9080/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
"plugins": {
"fault-injection": {
"abort": {
"http_status": 500,
"body": "Fault Injection!\n"
}
}
},
"uri": "/hello"
}'
```

测试:

```shell
$ curl http://127.0.0.1:9080/hello -v
......
< HTTP/1.1 500 Internal Server Error
< Date: Wed, 25 Nov 2020 14:50:20 GMT
< Content-Type: text/plain; charset=utf-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< Server: APISIX/2.0
<
Fault Injection!
```

没有 `X-APISIX-Upstream-Status` 的响应头。

示例3:`Upstream` 具有多节点,并且所有节点不可用

```shell
$ curl http://127.0.0.1:9080/apisix/admin/upstreams/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
"nodes": {
"127.0.0.3:1": 1,
"127.0.0.2:1": 1,
"127.0.0.1:1": 1
},
"retries": 2,
"type": "roundrobin"
}'
```

```shell
$ curl http://127.0.0.1:9080/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
"uri": "/hello",
"upstream_id": "1"
}'
```

测试:

```shell
$ curl http://127.0.0.1:9080/hello -v
< HTTP/1.1 502 Bad Gateway
< Date: Wed, 25 Nov 2020 15:07:34 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 154
< Connection: keep-alive
< Server: APISIX/2.0
< X-APISIX-Upstream-Status: 502, 502, 502
<
<html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>openresty</center>
</body>
</html>
```

具有 `X-APISIX-Upstream-Status: 502, 502, 502` 的响应头。
5 changes: 5 additions & 0 deletions t/lib/server.lua
Original file line number Diff line number Diff line change
Expand Up @@ -365,4 +365,9 @@ function _M.log()
end


function _M.server_error()
error("500 Internal Server Error")
end

Firstsawyou marked this conversation as resolved.
Show resolved Hide resolved

return _M
Loading