Multiple source files

[dronelektron]

Help us help you

• I have checked that my issue doesn't exist yet.
• I have tried my absolute best to reduce the problem-space and have provided the absolute smallest test-case possible.
• I can always reproduce the issue with the provided description below.

Environment

• Operating System version: Windows 10 Pro
• Game/AppID (with version if applicable): 300
• Current SourceMod version: 1.11.0.6943
• Current SourceMod snapshot: 1.12.0.7114
• Current Metamod: Source snapshot: 1.11.0-dev+1153

• I have updated SourceMod to the latest version and it still happens.
• I have updated SourceMod to the latest snapshot and it still happens.
• I have updated SourceMM to the latest snapshot and it still happens.

Description

A project consisting of multiple source files either does not compile or produces a warning.

Usage: spcomp.exe [options] <filename> [filename...]

Problematic Code (or Steps to Reproduce)

File: plugin.sp

#include <sourcemod>

public Plugin myinfo = {
    name = "",
    author = "",
    description = "",
    version = "",
    url = ""
};

public void OnPluginStart() {
    Module_Foo();
}

File: module.sp

void Module_Foo() {
    PrintToServer("Foo from module");
}

Logs

drone@ANDY-PC MINGW64 ~/Desktop/multiple-sources
$ spcomp -o plugin.smx plugin.sp module.sp
SourcePawn Compiler 1.12.0.7114
Copyright (c) 1997-2006 ITB CompuPhase
Copyright (c) 2004-2021 AlliedModders LLC

core.sp(12) : error 017: undefined symbol "Module_Foo"
    12 |     Module_Foo();
-------------^

1 Error.

If you change the order of the source files to module.sp plugin.sp, you will get the following:

$ spcomp -o plugin.smx module.sp plugin.sp
SourcePawn Compiler 1.12.0.7114
Copyright (c) 1997-2006 ITB CompuPhase
Copyright (c) 2004-2021 AlliedModders LLC

module.sp(1) : warning 203: symbol is never used: "Module_Foo"
     1 | void Module_Foo() {
--------------^
Code size:         3488 bytes
Data size:         2280 bytes
Stack/heap size:      16452 bytes
Total requirements:   22220 bytes

1 Warning.

If you add the public keyword before the Module_Foo function, the project will compile, but the function will not be called:

public void Module_Foo() {
    PrintToServer("[DEBUG] Foo from module");
}

[dvander]

This feature was dropped a while ago. I can look into re-adding it.

[dronelektron]

Sadly. I think it would be good to re-add it if it does not complicate the code base too much.

I like C style projects, when you have only headers in source files, but source files are attached to the compiler for linking.

This will hide the long and scary paths to the source files of other plugins, and make the build process more flexible.

[dvander]

My main motivation here is that it's better than having a "main.sp" that includes a bunch of source files. Also, many modern languages treat same-dir files as automatically part of a single package/module. It seems like a natural direction to go.

The reason it got removed was during the transition to the new parser. That transition is long done, so we should be able to re-add this.

[dvander]

This is going to be harder than I thought. The old compiler would concatenate all the source files together, which meant they had to be correctly ordered on the command line. I don't want to do that. Each source file should compile independently and then get linked together, which also means they should each get their own global scope, and have to re-import includes. Unfortunately... we don't have a linker, and I don't want to write one, since that's neither a useful nor a productive use of time.

So I'll think about this a bit more.

[dvander]

Most likely the real answer is to finally write a replacement for #include, which multi-source projects would be obliged to switch to.

[dronelektron]

Thank you for taking the time to look into this issue. It's sad that the language has everything except a linker. It would be very cool to turn into a normal scripting language without boilerplate code, where each file can be treated as a module with its own constants, functions and more.

[dvander]

Spitballing a bit, I can see this going in 3 directions.

(1) Runtime linking. Compile every file into its own .smx - including stuff like sourcemod.inc. The compiler would put these all into a single container. The VM would open the container, and link everything together in memory. This is how Java/C# work (.jar files are just a zip of .class binaries). It would enable a run-as-source model. It'd be moderate changes to spcomp and heavy changes to the VM.

(2) Compile-time linking, with AST merging. We'd build separate in-memory ASTs and symbol tables of every file, aggregate them together, then generate one big .smx file, as if they were never separate to begin with. It'd be heavy changes to spcomp and no changes to the VM.

(3) Compile-time linking, with SMX merging. We'd do all the stages of compilation, then do the runtime linking from (1) that would have been in the VM, then write it out. Probably the most work of any option, but wouldn't require as invasive refactoring of the compiler as (2).

I'll think about it for a little while.