chore(*) Bundling proto files inside the luarock

[tyler-ball]

Summary

This allows us to incloud these source files as siblings to the rest of
our source code. It also allows us to look up the location of them
at runtime instead of hardcoding them into /usr/local/kong/include.

This does mean future proto files (or other includes) will need to get added to the .rockspec but I think that is better than hardcoding the protobuf lookup path.

Full changelog

• Bundle our proto files inside the Kong luarock

Related

• fix(proto) move all proto files to /usr/local/kong/include #8914
• https://github.com/Kong/kong-ee/pull/3120#discussion_r854275965
• feat(*) We are bundling our proto files into the Kong luarock  kong-build-tools#484

[tyler-ball]

Depends on Kong/kong-build-tools#484

[tyler-ball]

loader.which returns the path to a specified module

[mayocream]

Add a test to load the .proto file with pure luarocks loader.

Suggested change

	end)
	end)
	it("loads .proto files with luarocks loader", function()
	local loader = require "luarocks.loader"
	local kong_init_file, _, _ = loader.which("kong")
	local kong_include_dir = kong_init_file:gsub("init%.lua$", "include")
	local protoc = require "protoc"
	local p = protoc.new()
	p.include_imports = true
	p:addpath(kong_include_dir)
	assert(p:loadfile("opentelemetry/proto/collector/trace/v1/trace_service.proto"))
	end)

[kikito]

Overprotective code is not my cup of tea, but this worries me. Two concerns:

• The GRPC module shouldn't "know" about kong internal structure (like that our "main luarock uses a file called init.lua")
• If we for some reason decide to rename init.lua to something else (unlikely, but it might happen if we decide to change how we load modules in the far future), this will fail with an obscure message.

My first impulse would be moving this logic to a function in kong.tools.utils.get_include_module and also write a test for it, so it breaks the tests if we change things. However I believe the grpc module gets required before utils, so this might not be possible. I don't know where else I would put this function. Kong.global or even kong.constants, perhaps? The path shouldn't change once kong starts after all.

[tyler-ball]

• The GRPC module needs to know that from the root of the installed luarock (or source code directory), there is a folder named kong/include that stores all the proto files to be loaded. I agree that looking for kong/init.lua isn't the best approach but there isn't a luarock API right now for "give me the install root of a rock", only "give me the path to a file in a luarock").
• I'm against moving tiny pieces of code that are only used once into utility modules. I think that just obfuscates the code and makes it harder to read. I did add a new comment here to (hopefully) better explain why I am calling the current API.
• I think if we renamed kong/init.lua the tests I wrote would break trying to call :gsub on a nil table. I'm open to adding an assert with a better message like assert(kong_init_file, "Could not find the 'init.lua' file in the root of the kong luarock"). Thoughts?

[kikito]

Let's go with the assert then.

Suggested change

	local kong_include_dir = kong_init_file:gsub("init%.lua$", "include")
	local kong_include_dir = kong_init_file:gsub("init%.lua$", "include")
	assert(kong_include_dir, "Could not find the 'init.lua' file in the root of the kong luarock")

[Tieske]

drive by comment:

have we been been looking at https://github.com/hishamhm/datafile for this functionality?

It allows to load files included in a rockspec like this: https://github.com/hishamhm/datafile/blob/78efd1f9de2c15a00c7d7c73ea71684c0d50a2f4/test/test-datafile-scm-1.rockspec#L24-L26

[hbagdi]

Closing this due to lack of activity. Please re-open if needed.