Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redis Cluster Proxy Crash When I write redis data in batches #114

Open
JackDan9 opened this issue Mar 3, 2023 · 0 comments
Open

Redis Cluster Proxy Crash When I write redis data in batches #114

JackDan9 opened this issue Mar 3, 2023 · 0 comments

Comments

@JackDan9
Copy link

JackDan9 commented Mar 3, 2023

[2023-03-03 11:46:20.643/M] Redis Cluster Proxy v999.999.999 (unstable)
[2023-03-03 11:46:20.643/M] Commit: (ac83840/1)
[2023-03-03 11:46:20.643/M] Git Branch: unstable
[2023-03-03 11:46:20.643/M] PID: 1
[2023-03-03 11:46:20.643/M] OS: Linux 3.10.0-957.el7.x86_64 x86_64
[2023-03-03 11:46:20.643/M] Bits: 64
[2023-03-03 11:46:20.643/M] Log level: error
[2023-03-03 11:46:20.643/M] Connections pool size: 10 (respawn 2 every 50ms if below 10)
[2023-03-03 11:46:20.644/M] Starting 8 threads...
[2023-03-03 11:46:20.644/M] Fetching cluster configuration...
[2023-03-03 11:46:20.647/M] Cluster Address: redis-cluster:6379
[2023-03-03 11:46:20.647/M] Cluster has 3 masters and 3 replica(s)
[2023-03-03 11:46:20.731/M] All thread(s) started!
[2023-03-03 11:49:55.003/6] Thread 0 terminated
[2023-03-03 11:49:55.003/6] Thread 1 terminated
[2023-03-03 11:49:55.003/6] Thread 2 terminated
[2023-03-03 11:49:55.003/6] Thread 3 terminated
[2023-03-03 11:49:55.003/6] Thread 4 terminated
[2023-03-03 11:49:55.003/6] Thread 5 terminated
[2023-03-03 11:49:55.003/6] Thread 7 terminated

=== PROXY BUG REPORT START: Cut & paste starting from here ===
[2023-03-03 11:49:55.003/6] Redis Cluster Proxy 999.999.999 crashed by signal: 11
[2023-03-03 11:49:55.003/6] Crashed running the instruction at: 0x40831c
[2023-03-03 11:49:55.003/6] Accessing address: 0x1c0
[2023-03-03 11:49:55.003/6] Handling crash on thread: 6

------ STACK TRACE ------
EIP:
redis-cluster-proxy(listDelNode+0xc)[0x40831c]

Backtrace:
redis-cluster-proxy(logStackTrace+0x2d)[0x40d0fd]
redis-cluster-proxy(sigsegvHandler+0x186)[0x40d716]
/lib64/libpthread.so.0(+0xf630)[0x7f1538239630]
redis-cluster-proxy(listDelNode+0xc)[0x40831c]
redis-cluster-proxy[0x410d2f]
redis-cluster-proxy[0x419d88]
redis-cluster-proxy(aeProcessEvents+0x13b)[0x408d7b]
redis-cluster-proxy(aeMain+0x2b)[0x40913b]
redis-cluster-proxy[0x410f9c]
/lib64/libpthread.so.0(+0x7ea5)[0x7f1538231ea5]
/lib64/libc.so.6(clone+0x6d)[0x7f1537f5a96d]

------ INFO OUTPUT ------

Proxy

proxy_version:999.999.999
proxy_git_sha1:ac83840d
proxy_git_dirty:1
proxy_git_branch:unstable
os:Linux 3.10.0-957.el7.x86_64 x86_64
arch_bits:64
multiplexing_api:epoll
gcc_version:7.3.1
process_id:1
threads:8
tcp_port:7777
uptime_in_seconds:215
uptime_in_days:0
config_file:/data/proxy.conf
acl_user:default

Memory

used_memory:17357512
used_memory_human:16.55M
total_system_memory:405186953216
total_system_memory_human:377.36G

Clients

connected_clients:6
max_clients:10000
thread_0_clinets:1
thread_1_clinets:1
thread_2_clinets:1
thread_3_clinets:0
thread_4_clinets:1
thread_5_clinets:1
thread_6_clinets:1
thread_7_clinets:0

Cluster

address:
entry_node::0

---- SIZEOF STRUCTS ----
clientRequest: 184
client: 224
redisClusterConnection: 48
clusterNode: 112
redisCluster: 104
list: 48
listNode: 24
rax: 24
raxNode: 4
raxIterator: 480
aeEventLoop: 88
aeFileEvent: 32
aeTimeEvent: 64

------ REGISTERS ------

RAX:000000000109ae70 RBX:00000000000001c0
RCX:00007f15382237b8 RDX:0000000000000006
RDI:000000000109aeb0 RSI:00000000000001c0
RBP:000000000109aeb0 RSP:00007f15347cddd0
R8 :0000000000000046 R9 :000000000042be54
R10:0000000000000004 R11:373736310a0d3031
R12:0000000000000000 R13:000000000109aeb0
R14:000000000108adc0 R15:00000000010acfa0
RIP:000000000040831c EFL:0000000000010202
CSGSFS:0000000000000033
(00007f15347cdddf) -> 00007f1518261570
(00007f15347cddde) -> 0000000000000001
(00007f15347cdddd) -> 0000000000000143
(00007f15347cdddc) -> 00000000010acfa4
(00007f15347cdddb) -> 0000000000000143
(00007f15347cddda) -> 000000000118d3f0
(00007f15347cddd9) -> 000000061825ee60
(00007f15347cddd8) -> 0000000000000000
(00007f15347cddd7) -> 0000000000419ea8
(00007f15347cddd6) -> 000000000042b45d
(00007f15347cddd5) -> 0000000000419d88
(00007f15347cddd4) -> 00007f151818e6a0
(00007f15347cddd3) -> 0000000000410d2f
(00007f15347cddd2) -> 00007f1518261570
(00007f15347cddd1) -> 00007f151818e6a0
(00007f15347cddd0) -> 00007f1518261570

------ DUMPING CODE AROUND EIP ------
Symbol: listDelNode (base: 0x408310)
Module: redis-cluster-proxy (base 0x400000)
$ xxd -r -p /tmp/dump.hex /tmp/dump.bin
$ objdump --adjust-vma=0x408310 -D -b binary -m i386:x86-64 /tmp/dump.bin

dump of function (hexdump of 140 bytes):
55534889fd4889f34883ec08488b16488b46084885d2743848894208488b46084885c07433488910488b45184885c07406488b7b10ffd04889dfe801ad010048836d28014883c4085b5dc30f1f4400004885c048890775cd48895508ebca669055534889fdbf1000000089f34883ec08e8bbab01004885c0740e85db7412488b55084889108958084883c408
Function at 0x423050 is zfree
Function at 0x422f40 is zmalloc

=== PROXY BUG REPORT END. Make sure to include from START to END. ===

   Please report the crash by opening an issue on github:

       https://github.com/artix75/redis-cluster-proxy/issues
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant