Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reduce collectgarbage() time cost in tl build #579

Closed
virusdefender opened this issue Nov 4, 2022 · 1 comment
Closed

reduce collectgarbage() time cost in tl build #579

virusdefender opened this issue Nov 4, 2022 · 1 comment

Comments

@virusdefender
Copy link

virusdefender commented Nov 4, 2022

My project contains about 100+ tl files, the code added shows collectgarbage in tl build will cost about 70% cpu time.

before gc memory	loop 2	47309.854492188
after  gc memory	loop 2	19262.390625
......

before gc memory	loop 30	19282.856445312
after  gc memory	loop 30	19226.875976562
......

before gc memory	loop 100	28070.041992188
after  gc memory	loop 100	28028.818359375
......

before gc memory	loop 128	35244.404296875
after  gc memory	loop 128	35152.580078125

build total time	2626.44
gc total time	1870.631
diff --git a/tl b/tl
index 1dc9a8e..ba0123f 100755
--- a/tl
+++ b/tl
@@ -430,6 +430,8 @@ do
    end
 
    function build.run(tlconfig)
+      local build_start = os.clock()
+      local gc_time = 0
       local function remove_leading_path(leading_part, path)
          local s, e = path:find("^" .. leading_part .. PATH_SEPARATOR .. "?")
          if s then
@@ -718,11 +720,17 @@ do
          end
 
          if i > 1 then
+            local a = os.clock()
+            print("before gc memory", "loop " .. tostring(i), collectgarbage("count"))
             collectgarbage()
+            print("after  gc memory", "loop " .. tostring(i), collectgarbage("count"))
+            gc_time = gc_time + (os.clock() - a)
          end
       end
 
       local ok = report_all_errors(tlconfig, env)
+      print("build total time", (os.clock() - build_start) * 1000)
+      print("gc total time", gc_time * 1000)
 
       os.exit(ok and 0 or 1)
    end

if if i > 1 then is replaced by if i % 50 == 0 (gc once per 50 files), the time cost is reduced significantly and the memory usage is acceptable.

before gc memory	loop 50	50841.455078125
after  gc memory	loop 50	19465.637695312

before gc memory	loop 100	46594.625976562
after  gc memory	loop 100	28092.53125

build total time	1107.268
gc total time	96.925
@hishamhm
Copy link
Member

hishamhm commented Nov 7, 2022

tl build is currently deprecated, and I intend to remove this command from the core tl compiler it in a future release; I recommend using cyan instead as a build tool — does the same issue happen if building using cyan build?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants