Lispy mode interferes with evil-escape

[Sleepful]

this is my config:

(after! evil-escape (progn
                      (setq-default evil-escape-key-sequence "jk")
                      (setq-default evil-escape-unordered-key-sequence t)))

but lispy mode has keybindings for insert mode and it seems to capture the keys without letting them reach evil-escape.
Any way to work around the issue?