Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sharedata got stack overflow when 100w string key in use #1981

Open
spin6lock opened this issue Oct 8, 2024 · 2 comments
Open

sharedata got stack overflow when 100w string key in use #1981

spin6lock opened this issue Oct 8, 2024 · 2 comments

Comments

@spin6lock
Copy link
Contributor

--examples/sharedata_size.lua
local skynet = require "skynet"
local sharedata = require "skynet.sharedata"

local big_table = {}
local size = 1000000
for i=1,size do
        big_table[tostring(i)] = i
end

skynet.start(function()
        skynet.error("new foobar")
        sharedata.new("foobar", big_table)
end)

跑这个样例的时候会报错StackOverflow

[:01000004] master listen socket 0.0.0.0:2013
[:01000005] LAUNCH snlua cslave
[:01000005] slave connect to master 127.0.0.1:2013
[:01000004] connect from 127.0.0.1:33644 4
[:01000006] LAUNCH harbor 1 16777221
[:01000004] Harbor 1 (fd=4) report 127.0.0.1:2526
[:01000005] Waiting for 0 harbors
[:01000005] Shakehand ready
[:01000007] LAUNCH snlua datacenterd
[:01000008] LAUNCH snlua service_mgr
[:01000009] LAUNCH snlua sharedata_size
[:01000009] Memory warning 32.00 M
[:01000009] Memory warning 64.00 M
[:0100000a] LAUNCH snlua sharedatad
[:01000009] new foobar
[:0100000a] Memory warning 32.00 M
[:0100000a] Memory warning 64.00 M
[:0100000a] lua call [1000009 to :100000a : 3 msgsz = 11757584] error : ./lualib/skynet.lua:988: ./lualib/skynet.lua:452: stack overflow
stack traceback:
        [C]: in function 'skynet.sharedata.core.new'
        ./service/sharedatad.lua:15: in upvalue 'newobj'
        ./service/sharedatad.lua:76: in local 'f'
        ./service/sharedatad.lua:162: in upvalue 'f'
        ./lualib/skynet.lua:402: in function <./lualib/skynet.lua:374>
stack traceback:
        [C]: in function 'assert'
        ./lualib/skynet.lua:988: in function 'skynet.dispatch_message'
[:01000009] init service failed: ./lualib/skynet.lua:720: call failed
stack traceback:
        [C]: in function 'error'
        ./lualib/skynet.lua:720: in upvalue 'yield_call'
        ./lualib/skynet.lua:737: in function 'skynet.call'
        ./lualib/skynet/sharedata.lua:45: in function 'skynet.sharedata.new'
        ./examples/sharedata_size.lua:12: in upvalue 'start'
        ./lualib/skynet.lua:1065: in function <./lualib/skynet.lua:1063>
        [C]: in function 'xpcall'
        ./lualib/skynet.lua:1067: in function 'skynet.init_service'
        ./lualib/skynet.lua:1080: in upvalue 'f'
        ./lualib/skynet.lua:375: in function <./lualib/skynet.lua:374>
[:01000009] KILL self
[:01000002] KILL self

注释掉 lualib-src/lua-sharedata.c:373能好,luaL_checkstack应该在使用lua栈之前用;但是有同学觉得lualib-src/lua-sharedata.c:383的lua_settop会导致栈顶设置异常,大家有什么建议呢?

@cloudwu
Copy link
Owner

cloudwu commented Oct 9, 2024

这里设计上是用 lua stack 当数组使用来保存一个 id 到 string 的映射表。并没有考虑超过 100 万的情况。
如果不重新实现,那么你需要修改 luaconf.h 里的 LUAI_MAXSTACK ,默认就是 100 万。

否者,你得重新实现这个映射表。比较接近的方案是使用 userdata 的 uservalue 来保存这张表,应该有更大的限制。
lua_getiuservalue 会略微慢于 lua_value

https://github.com/cloudwu/skynet/blob/master/lualib-src/lua-sharedata.c#L412 这里在 L 里面放一个足够大的 userdata 。然后用 lua_setiuservalue 把 string 放进去。

或者用一个 table 也可以,lua_rawgeti 会比 lua_getiuservalue 更慢一丁点。用 table 的好处是容量和 stack 一样是弹性的。userdata 的 uservalue 个数创建时就需要决定好。

@spin6lock
Copy link
Contributor Author

我们先分离下配置表,搞多几个入口

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants