Suggestions for the structure of the project moving forward

[euclidianAce]

I thought of some of these ideas while working on the build options (#177) and would like to know your opinions. These are definitely more abstract and long-term oriented goals, but I think they're worth discussing :)

• Further separate the compiler (tl.tl) from the cli (tl)
  • As in make them separate modules. For example:
    • luarocks install Teal: pure lua, no dependencies, just the minimal compiler api
    • luarocks install tealc: pulls in Teal, argparse, lfs, and any future bells and whistles we may add
    • (These names could definitely be confusing when someone does a luarocks search teal, maybe change Teal to tealstandalone or something and just have teal pull in everything)
• With the cli completely (for the most part) separate from the compiler, people who use different runtimes of lua (where C related dependencies are unavailable, or maybe the implementation is incomplete in such a way that the compiler works but the cli doesn't) can write their own minimal compilers in like 20 lines¹ of code with the api
  • Additionally the cli could have its own api, my thinking here is if we expose an api to the official compiler, luarocks (and maybe other package managers) could hook into it to ease writing modules
    • Specifying an api could simplify the part of the code that actually does the compiling
    • Integrating into other build systems could be made easier
      • We could probably look to Tup for some inspiration, since it embeds a Lua interpreter to write Tupfiles

Some of these came up when thinking about use cases for Teal and one that came to mind were the Minecraft mods ComputerCraft and OpenComputers, which embed LuaJ and Lua respectively. (I think OpenComputers falls back to LuaJ if it can't use a regular interpreter). On these restricted platforms, people write suprisingly sizable projects² but wouldn't have access to modules like lfs due to the filesystem being handled differently internally.

I don't know how much other people would want this or if there are any glaring flaws in my logic that I'm missing, but I'd definitely have a usecase for it and would definitely be willing to write the prs.

²: I'd say OpenOS, the default OS for OpenComputers is quite impressive, and when the stars align it can run on actual hardware with a modified Linux kernel.
¹: ~20 lines for a decently functional bare-bones compiler

#!/usr/bin/env lua
local input, output = ...
assert(input)
output = output or input:gsub("%.tl$", ".lua")
local tl = require("tl")
local res = assert(tl.process(input))
local function report_errors(category, errors)
	if #errors == 0 then return end
	for _, e in ipairs(errors) do
		io.stderr:write(category, ":", e.filename, ":", e.y, ":", e.x, ": ", (e.msg or ""), "\n")
	end
	os.exit(1)
end
report_errors("syntax", res.syntax_errors)
report_errors("unknowns", res.unknowns)
report_errors("type", res.type_errors)
local ofd = assert(io.open(output, "w"))
ofd:write(tl.pretty_print_ast(res.ast) .. "\n")
ofd:close()
print("Wrote: " .. output)

[hishamhm]

Further separate the compiler (tl.tl) from the cli (tl), as in make them separate modules

Agree! I've been thinking about this. The names are something to figure out later, but yes, I'd like to have exactly what you described: a no-dependencies pure Lua compiler that can be embedded into any kind of (compatible) environment, and a convenient top-level CLI package with all the bells and whistles.