Levels with lots of random string concentration lag a lot with the ui

[Bauumm]

When the game uses a lot of memory (unrelated to the level) any level that concentrates random characters to a string a lot will start lagging a lot.

The only part I could reproduce in a minimal example is string concentration becoming slow with more memory usage, but in the game this only seems to happen when the added characters are randomized.
Here is said minimal example:

-- this part allocates a lot of useless memory
S = {}
for _ = 1, 1000 do
    local t = {}
    for _ = 1, 10000 do
        t[{}] = {}
    end
    S[{}] = t
end

-- this part concentrats a string and prints the time the operation took
local uv = require("luv")
local start = uv.hrtime()
local t = ""
for _ = 1, 10 ^ 5 do
    t = t .. "1"
end
print(uv.hrtime() - start)

on my pc the result is 6906122093 nanoseconds, if i remove the useless allocation at the top the result is 244013567 nanoseconds so more than 28 times as fast.

The issue in the game does not happen when running a replay as it bypasses the initial loading of the ui which makes it allocate less memory initially. However as I said earlier changing the level script to use a hardcoded character instead of a random one seems to make the issue go away, so I'm still not certain about the cause.

Any help with finding why this happens or even a way to fix it would be appreciated.

[ColonelThirtyTwo]

It's a well known issue with Lua's string concatentation. Since Lua strings are immutable, concatenating a string requires allocating a new string every time.

Instead, append the substrings to a table and then use table.concat at the very end.

[Bauumm]

Well that's what I would do if I wrote that code, but this is about a user made level that works just fine in older versions of the game (that were written in c++). So ideally oh3 would be able to replicate that.

There are 2 main approaches i could think of for fixing this:

1. put the levels lua stuff in another thread or process to allow it to have its own memory
2. somehow find a way to replace costly string operations in a level's script automatically (e.g. by reimplementing the string type with metatables or by analyzing the code and automatically simplifying it)

All of these ideas sound pretty annoying to implement and I don't even know if they would actually fix it which is why I have not dealt with this.