Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backport THREESCALE-10167 cpu detection to 2.14 #1412

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,12 @@ and this project adheres to [Semantic Versioning](http://semver.org/).

## [Unreleased]

### Added

- Detect number of CPU shares when running on Cgroups V2 [PR #1410](https://github.com/3scale/apicast/pull/1410) [THREESCALE-10167](https://issues.redhat.com/browse/THREESCALE-10167)

## [3.14.0] 2023-07-25

### Fixed

- In boot mode on `init_worker` check configuration expiration [PR #1399](https://github.com/3scale/APIcast/pull/1399) [THREESCALE-9003](https://issues.redhat.com/browse/THREESCALE-9003)
Expand Down
44 changes: 30 additions & 14 deletions gateway/src/apicast/cli/environment.lua
Original file line number Diff line number Diff line change
Expand Up @@ -43,36 +43,52 @@
end
end

local function detect_kubernetes()
local secrets = open('/run/secrets/kubernetes.io')
-- CPU shares in Cgroups v1 or converted from weight in Cgroups v2 in millicores
local function cpu_shares()
local shares

if secrets then secrets:close() end
-- This check is from https://github.com/kubernetes/kubernetes/blob/release-1.27/test/e2e/node/pod_resize.go#L305-L314
-- alternatively, this method can be used: https://kubernetes.io/docs/concepts/architecture/cgroups/#check-cgroup-version
-- (`stat -fc %T /sys/fs/cgroup/` returns `cgroup2fs` or `tmpfs`)
if pl_path.exists("/sys/fs/cgroup/cgroup.controllers") then
-- Cgroups v2
ngx.log(ngx.DEBUG, "detecting cpus in Cgroups v2")

Check warning on line 55 in gateway/src/apicast/cli/environment.lua

View check run for this annotation

Codecov / codecov/patch

gateway/src/apicast/cli/environment.lua#L55

Added line #L55 was not covered by tests
-- Using the formula from https://github.com/kubernetes/kubernetes/blob/release-1.27/pkg/kubelet/cm/cgroup_manager_linux.go#L570-L574
local file = open('/sys/fs/cgroup/cpu.weight')

Check warning on line 57 in gateway/src/apicast/cli/environment.lua

View check run for this annotation

Codecov / codecov/patch

gateway/src/apicast/cli/environment.lua#L57

Added line #L57 was not covered by tests

return secrets or resty_env.value('KUBERNETES_PORT')
end

local function cpu_shares()
if not detect_kubernetes() then return end
if file then
local weight = file:read('*n')
file:close()

Check warning on line 61 in gateway/src/apicast/cli/environment.lua

View check run for this annotation

Codecov / codecov/patch

gateway/src/apicast/cli/environment.lua#L59-L61

Added lines #L59 - L61 were not covered by tests

local shares
local file = open('/sys/fs/cgroup/cpu/cpu.shares')
shares = (((weight - 1) * 262142) / 9999) + 2

Check warning on line 63 in gateway/src/apicast/cli/environment.lua

View check run for this annotation

Codecov / codecov/patch

gateway/src/apicast/cli/environment.lua#L63

Added line #L63 was not covered by tests
end
else
-- Cgroups v1
ngx.log(ngx.DEBUG, "detecting cpus in Cgroups v1")
local file = open('/sys/fs/cgroup/cpu/cpu.shares')

if file then
shares = file:read('*n')
if file then
shares = file:read('*n')

file:close()
file:close()
end
end

return shares
end

local function cpus()
local shares = cpu_shares()
if shares then return ceil(shares / 1024) end
if shares then
local res = ceil(shares / 1024)
ngx.log(ngx.DEBUG, "cpu_shares = "..res)
return res
end

-- TODO: support /sys/fs/cgroup/cpuset/cpuset.cpus
-- see https://github.com/sclorg/rhscl-dockerfiles/blob/ff912d8764af9a41096e63064bbc325395afa608/rhel7.sti-base/bin/cgroup-limits#L55-L75
local nproc = util.system('nproc')
ngx.log(ngx.DEBUG, "cpus from nproc = "..nproc)

Check warning on line 91 in gateway/src/apicast/cli/environment.lua

View check run for this annotation

Codecov / codecov/patch

gateway/src/apicast/cli/environment.lua#L91

Added line #L91 was not covered by tests
return tonumber(nproc)
end

Expand Down
Loading