OPAL Server doesn't clean up symbolic links when github is down #634

kreyyser · 2024-08-06T08:41:16Z

When opal server has github policies setup and github is down for some time opal server seems like spawn zombie processes but apparently looks like it is just a list of symbolic links that are not cleaned up.

To Reproduce
run OPAL with github policies source as a container
make somehow github return 500
list processes in opal server container

Expected behavior
No zombie processes or broken links proc directory

Screenshots

OPAL version

Version: 0.7.6

gemanor · 2024-09-18T10:30:33Z

/bounty $100

algora-pbc · 2024-09-18T10:30:39Z

💎 $100 bounty • Permit.io

Steps to solve:

Start working: Comment /attempt #634 with your implementation plan
Submit work: Create a pull request including /claim #634 in the PR body to claim the bounty
Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

Thank you for contributing to permitio/opal!

Add a bounty • Share on socials

sroopsai · 2024-09-20T11:30:40Z

Hi @gemanor I'm interested in this task. Can I be assigned? Thank You.

onyedikachi-david · 2024-09-20T18:07:34Z

@gemanor I am trying to replicate this; seems to be working fine;

Replicating logs using toxiproxy

┌─[onyedikachi@Davids-MacBook-Air] - [~/Documents/codes/algora-bounties/opal] - [Fri Sep 20, 18:36]
└─[$] <git:(master)> docker exec toxiproxy wget -O- --post-data='{"name": "github", "listen": "0.0.0.0:8666", "upstream": "github.com:443"}' --header='Content-Type: application/json' http://localhost:8474/proxies
Connecting to localhost:8474 (127.0.0.1:8474)
{"name":"github","listen":"[::]:8666","upstream":"github.com:443","enabled":true,"toxics":[]}-                    100% |*******************************|    93   0:00:00 ETA

┌─[onyedikachi@Davids-MacBook-Air] - [~/Documents/codes/algora-bounties/opal] - [Fri Sep 20, 18:38]
└─[$] <git:(master)> docker exec toxiproxy wget -O- --post-data='{"type": "latency", "attributes": {"latency": 5000}}' --header='Content-Type: application/json' http://localhost:8474/proxies/github/toxics
Connecting to localhost:8474 (127.0.0.1:8474)
-                    100%{"attributes":{"latency":5000,"jitter":0},"name":"latency_downstream","type":"latency","stream":"downstream","toxicity":1} |*******************************|   122   0:00:00 ETA

┌─[onyedikachi@Davids-MacBook-Air] - [~/Documents/codes/algora-bounties/opal] - [Fri Sep 20, 18:38]
└─[$] <git:(master)> docker exec toxiproxy wget -O- --post-data='{"type": "timeout", "attributes": {"timeout": 10000}}' --header='Content-Type: application/json' http://localhost:8474/proxies/github/toxics
Connecting to localhost:8474 (127.0.0.1:8474)
-                    100% |*******************************|   112   0:00:00 ETA

{"attributes":{"timeout":10000},"name":"timeout_downstream","type":"timeout","stream":"downstream","toxicity":1}%                                                                                    
┌─[onyedikachi@Davids-MacBook-Air] - [~/Documents/codes/algora-bounties/opal] - [Fri Sep 20, 18:38]
└─[$] <git:(master)> docker exec toxiproxy wget -O- http://localhost:8474/proxies
Connecting to localhost:8474 (127.0.0.1:8474)
-                    100%{"github":{"name":"github","listen":"[::]:8666","upstream":"github.com:443","enabled":true,"toxics":[{"attributes":{"latency":5000,"jitter":0},"name":"latency_downstream","type":"latency","stream":"downstream","toxicity":1},{"attributes":{"timeout":10000},"name":"timeout_downstream","type":"timeout","stream":"downstream","toxicity":1}]}} |*******************************|   339   0:00:00 ETA

┌─[onyedikachi@Davids-MacBook-Air] - [~/Documents/codes/algora-bounties/opal] - [Fri Sep 20, 18:38]
└─[$] <git:(master)> docker-compose -f docker/docker-compose-example.yml stop opal_server opal_client
[+] Stopping 2/2
 ✔ Container docker-opal_client-1  Stopped                                                                                                                                                     10.3s 
 ✔ Container docker-opal_server-1  Stopped                                                                                                                                                     10.3s 
┌─[onyedikachi@Davids-MacBook-Air] - [~/Documents/codes/algora-bounties/opal] - [Fri Sep 20, 18:39]
└─[$] <git:(master)> docker-compose -f docker/docker-compose-example.yml up -d opal_server opal_client
[+] Running 3/3
 ✔ Container docker-broadcast_channel-1  Running                                                                                       0.0s 
 ✔ Container docker-opal_server-1        Started                                                                                       0.4s 
 ✔ Container docker-opal_client-1        Started                                                                                       0.5s 
┌─[onyedikachi@Davids-MacBook-Air] - [~/Documents/codes/algora-bounties/opal] - [Fri Sep 20, 18:41]
└─[$] <git:(master*)> docker-compose -f docker/docker-compose-example.yml logs -f opal_server
opal_server-1  | + exec gunicorn -b 0.0.0.0:7002 -k uvicorn.workers.UvicornWorker --workers=4 -c ./gunicorn_conf.py opal_server.main:app -t 30 --keep-alive 5
opal_server-1  | [2024-09-20 17:41:57 +0000] [7] [INFO] Starting gunicorn 22.0.0
opal_server-1  | [2024-09-20 17:41:57 +0000] [7] [INFO] Listening at: http://0.0.0.0:7002 (7)
opal_server-1  | [2024-09-20 17:41:57 +0000] [7] [INFO] Using worker: uvicorn.workers.UvicornWorker
opal_server-1  | 2024-09-20 17:41:57.771 | WARNING  | __config__:when_ready:17 - Finished pre loading scopes...
opal_server-1  | [2024-09-20 17:41:57 +0000] [11] [INFO] Booting worker with pid: 11
opal_server-1  | [2024-09-20 17:41:57 +0000] [12] [INFO] Booting worker with pid: 12
opal_server-1  | [2024-09-20 17:41:57 +0000] [15] [INFO] Booting worker with pid: 15
opal_server-1  | 2024-09-20T17:41:58.012734+0000 | 12 | opal_common.monitoring.apm              | INFO  | DataDog APM disabled
opal_server-1  | 2024-09-20T17:41:58.012798+0000 | 11 | opal_common.monitoring.apm              | INFO  | DataDog APM disabled
opal_server-1  | 2024-09-20T17:41:58.013167+0000 | 11 | opal_common.monitoring.metrics          | INFO  | DogStatsD metrics disabled
opal_server-1  | 2024-09-20T17:41:58.013166+0000 | 12 | opal_common.monitoring.metrics          | INFO  | DogStatsD metrics disabled
opal_server-1  | 2024-09-20T17:41:58.014262+0000 | 11 | opal_server.server                      | INFO  | OPAL was not provided with JWT encryption keys, cannot verify api requests!
opal_server-1  | 2024-09-20T17:41:58.014262+0000 | 12 | opal_server.server                      | INFO  | OPAL was not provided with JWT encryption keys, cannot verify api requests!
opal_server-1  | 2024-09-20T17:41:58.014370+0000 | 11 | opal_server.pubsub                      | INFO  | Initializing broadcaster for server<->server communication
opal_server-1  | 2024-09-20T17:41:58.014372+0000 | 12 | opal_server.pubsub                      | INFO  | Initializing broadcaster for server<->server communication
opal_server-1  | 2024-09-20T17:41:58.045725+0000 | 12 | opal_server.server                      | INFO  | *** OPAL Server Startup ***
opal_server-1  | 2024-09-20T17:41:58.045734+0000 | 11 | opal_server.server                      | INFO  | *** OPAL Server Startup ***
opal_server-1  | 2024-09-20T17:41:58.045904+0000 | 11 | opal_server.server                      | INFO  | leadership lock acquired, leader pid: 11
opal_server-1  | 2024-09-20T17:41:58.045956+0000 | 11 | opal_common.topics.publisher            | INFO  | started broadcaster keepalive task: topic is '__broadcast_session_keepalive__', interval is 3600 seconds
opal_server-1  | 2024-09-20T17:41:58.046093+0000 | 11 | opal_server.policy.watcher.factory      | INFO  | Policy repo will be cloned to: /opal/regoclone/opal_repo_clone
opal_server-1  | 2024-09-20T17:41:58.046174+0000 | 11 | opal_server.policy.watcher.task         | INFO  | Launching policy watcher
opal_server-1  | 2024-09-20T17:41:58.046431+0000 | 11 | fastapi_websocket_pubsub.event_broadc...| INFO  | Listening for incoming events from broadcast channel (first listener started)
opal_server-1  | 2024-09-20T17:41:58.046487+0000 | 11 | opal_server.policy.watcher.task         | INFO  | listening on webhook topic: 'webhook'
opal_server-1  | 2024-09-20T17:41:58.048350+0000 | 11 | opal_common.git_utils.repo_cloner       | INFO  | Cloning repo from 'https://github.com/permitio/opal-example-policy-repo' to '/opal/regoclone/opal_repo_clone' (branch: 'master')
opal_server-1  | 2024-09-20T17:41:58.049119+0000 | 11 | fastapi_websocket_pubsub.event_broadc...| INFO  | Starting broadcaster listener
opal_server-1  | [2024-09-20 17:41:58 +0000] [17] [INFO] Booting worker with pid: 17
opal_server-1  | 2024-09-20T17:41:58.084883+0000 | 15 | opal_common.monitoring.apm              | INFO  | DataDog APM disabled
opal_server-1  | 2024-09-20T17:41:58.085484+0000 | 15 | opal_common.monitoring.metrics          | INFO  | DogStatsD metrics disabled
opal_server-1  | 2024-09-20T17:41:58.086213+0000 | 15 | opal_server.server                      | INFO  | OPAL was not provided with JWT encryption keys, cannot verify api requests!
opal_server-1  | 2024-09-20T17:41:58.086425+0000 | 15 | opal_server.pubsub                      | INFO  | Initializing broadcaster for server<->server communication
opal_server-1  | 2024-09-20T17:41:58.116861+0000 | 15 | opal_server.server                      | INFO  | *** OPAL Server Startup ***
opal_server-1  | 2024-09-20T17:41:58.195044+0000 | 17 | opal_common.monitoring.apm              | INFO  | DataDog APM disabled
opal_server-1  | 2024-09-20T17:41:58.195432+0000 | 17 | opal_common.monitoring.metrics          | INFO  | DogStatsD metrics disabled
opal_server-1  | 2024-09-20T17:41:58.195966+0000 | 17 | opal_server.server                      | INFO  | OPAL was not provided with JWT encryption keys, cannot verify api requests!
opal_server-1  | 2024-09-20T17:41:58.196045+0000 | 17 | opal_server.pubsub                      | INFO  | Initializing broadcaster for server<->server communication
opal_server-1  | 2024-09-20T17:41:58.198119+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:41:58.216269+0000 | 17 | opal_server.server                      | INFO  | *** OPAL Server Startup ***
opal_server-1  | 2024-09-20T17:41:58.476571+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:41:58.598549+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:41:59.099518+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:41:59.648438+0000 | 15 | fastapi_websocket_pubsub.event_broadc...| INFO  | Listening for incoming events from broadcast channel (first listener started)
opal_server-1  | 2024-09-20T17:41:59.648460+0000 | 17 | fastapi_websocket_pubsub.event_broadc...| INFO  | Listening for incoming events from broadcast channel (first listener started)
opal_server-1  | 2024-09-20T17:41:59.649088+0000 | 17 | fastapi_websocket_pubsub.event_broadc...| INFO  | Starting broadcaster listener
opal_server-1  | 2024-09-20T17:41:59.649092+0000 | 15 | fastapi_websocket_pubsub.event_broadc...| INFO  | Starting broadcaster listener
opal_server-1  | 2024-09-20T17:41:59.669025+0000 | 17 | fastapi_websocket_rpc.websocket_rpc_e...| INFO  | Client connected
opal_server-1  | 2024-09-20T17:41:59.669685+0000 | 17 | websockets.legacy.server                | INFO  | connection open
opal_server-1  | 2024-09-20T17:41:59.669957+0000 | 15 | fastapi_websocket_rpc.websocket_rpc_e...| INFO  | Client connected
opal_server-1  | 2024-09-20T17:41:59.670631+0000 | 15 | websockets.legacy.server                | INFO  | connection open
opal_server-1  | 2024-09-20T17:41:59.708588+0000 | 11 | opal_server.data.api                    | INFO  | Serving source configuration
opal_server-1  | 2024-09-20T17:41:59.708823+0000 | 12 | uvicorn.protocols.http.httptools_impl   | INFO  | 172.19.0.4:57610 - "GET /policy?path=. HTTP/1.1" 503
opal_server-1  | 2024-09-20T17:41:59.708903+0000 | 11 | uvicorn.protocols.http.httptools_impl   | INFO  | 172.19.0.4:57616 - "GET /data/config HTTP/1.1" 200
opal_server-1  | 2024-09-20T17:41:59.711237+0000 | 11 | opal_server.data.api                    |WARNING | Serving default all-data route, meaning DATA_CONFIG_SOURCES was not configured!
opal_server-1  | 2024-09-20T17:41:59.711377+0000 | 11 | uvicorn.protocols.http.httptools_impl   | INFO  | 172.19.0.4:57630 - "GET /policy-data HTTP/1.1" 200
opal_server-1  | 2024-09-20T17:42:00.433137+0000 | 17 | uvicorn.protocols.http.httptools_impl   | INFO  | 172.19.0.4:57632 - "GET /policy?path=. HTTP/1.1" 503
opal_server-1  | 2024-09-20T17:42:00.911339+0000 | 17 | uvicorn.protocols.http.httptools_impl   | INFO  | 172.19.0.4:57640 - "GET /policy?path=. HTTP/1.1" 503
opal_server-1  | 2024-09-20T17:42:02.333280+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:42:03.825769+0000 | 17 | uvicorn.protocols.http.httptools_impl   | INFO  | 172.19.0.4:58364 - "GET /policy?path=. HTTP/1.1" 503
opal_server-1  | 2024-09-20T17:42:09.837089+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:42:11.090002+0000 | 17 | uvicorn.protocols.http.httptools_impl   | INFO  | 172.19.0.4:58372 - "GET /policy?path=. HTTP/1.1" 503
opal_server-1  | 2024-09-20T17:42:11.801752+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:42:36.468977+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:42:40.567739+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:43:07.746330+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:43:14.849461+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:43:28.527766+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:43:48.992953+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:43:56.768192+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:44:06.267090+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:44:21.376191+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:44:49.010690+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:44:49.124212+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:45:08.329527+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:45:37.381865+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:46:00.265777+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:46:25.112950+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:46:45.171187+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:47:09.747848+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:47:21.355348+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:47:28.499200+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:47:44.470503+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:47:59.221237+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:48:27.461919+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:48:30.746226+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:48:31.105973+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:48:44.245960+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:48:47.811824+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:48:49.139385+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:49:08.729005+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:49:17.909007+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:49:33.220675+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:49:47.082229+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:50:11.703199+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:50:18.889581+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:50:22.436961+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:50:24.396999+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:50:44.178181+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:51:03.512839+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:51:09.938511+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:51:32.587447+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:51:45.621470+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:51:49.469047+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:51:57.782177+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:52:01.611626+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:52:29.540788+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:52:37.886343+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:53:03.061564+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:53:10.775901+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:53:11.052095+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:53:25.802246+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:53:29.603888+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:53:54.701828+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:54:08.243837+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:54:35.494441+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:54:54.073312+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:55:06.485363+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:55:09.990394+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:55:20.564042+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:55:33.882323+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:55:59.056847+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:56:13.184216+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:56:27.473293+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:56:50.704705+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:56:54.311478+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:57:03.650724+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:57:18.090909+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:57:40.700203+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:57:56.296209+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:58:14.987426+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:58:38.114188+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:58:51.931545+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:58:58.598166+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:59:05.920967+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:59:24.014810+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:59:24.795867+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T17:59:51.922849+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '
opal_server-1  | 2024-09-20T18:00:05.005612+0000 | 11 | opal_common.git_utils.repo_cloner       |ERROR  | cannot clone policy repo: Cmd('git') failed due to: exit code(128)
opal_server-1  |   cmdline: git clone -v --branch=master -- https://github.com/permitio/opal-example-policy-repo /opal/regoclone/opal_repo_clone
opal_server-1  |   stderr: 'Cloning into '/opal/regoclone/opal_repo_clone'...
opal_server-1  | fatal: unable to access 'https://github.com/permitio/opal-example-policy-repo/': gnutls_handshake() failed: Internal error
opal_server-1  | '

Checking proc for zombies:

> docker-compose -f docker/docker-compose-example.yml exec opal_server sh -c 'for pid in /proc/[0-9]*; do echo -n "${pid##*/} "; cat $pid/cmdline 2>/dev/null; echo; done'
1 bash./start.sh
11 /usr/local/bin/python/usr/local/bin/gunicorn-b0.0.0.0:7002-kuvicorn.workers.UvicornWorker--workers=4-c./gunicorn_conf.pyopal_server.main:app-t30--keep-alive5
12 /usr/local/bin/python/usr/local/bin/gunicorn-b0.0.0.0:7002-kuvicorn.workers.UvicornWorker--workers=4-c./gunicorn_conf.pyopal_server.main:app-t30--keep-alive5
15 /usr/local/bin/python/usr/local/bin/gunicorn-b0.0.0.0:7002-kuvicorn.workers.UvicornWorker--workers=4-c./gunicorn_conf.pyopal_server.main:app-t30--keep-alive5
17 /usr/local/bin/python/usr/local/bin/gunicorn-b0.0.0.0:7002-kuvicorn.workers.UvicornWorker--workers=4-c./gunicorn_conf.pyopal_server.main:app-t30--keep-alive5
294 sh-cfor pid in /proc/[0-9]*; do echo -n "${pid##*/} "; cat $pid/cmdline 2>/dev/null; echo; done
7 /usr/local/bin/python/usr/local/bin/gunicorn-b0.0.0.0:7002-kuvicorn.workers.UvicornWorker--workers=4-c./gunicorn_conf.pyopal_server.main:app-t30--keep-alive5

> docker-compose -f docker/docker-compose-example.yml exec opal_server ls -l /proc/*/exe
zsh: no matches found: /proc/*/exe

gemanor · 2024-09-21T19:53:09Z

@kreyyser can you please point @onyedikachi-david for reproduction steps?

@sroopsai let wait for @kreyyser comments to check the exact reproduction steps beforehand.

kreyyser · 2024-09-23T06:40:01Z

Hi @gemanor @sroopsai . I'll try to make a reproducible example this week.
To give more info - we have opal server in 2 replicas and kafka as a backbone. basically they work properly for some time with github and then it goes down and we observe these orphan symlinks. After opal server restart and github up - those links go away and do not appear until next github issue.

gemanor · 2024-09-30T08:02:28Z

@sroopsai / @kreyyser any updates?

kreyyser · 2024-09-30T11:31:16Z

Hi @gemanor,

sorry, haven't got a chance to try to reproduce this.
was focused last week more with this #671 as it affects us more.

I'll let you know the outcomes as soon as possible.

kreyyser added the bug Something isn't working label Aug 6, 2024

algora-pbc bot added the 💎 Bounty label Sep 18, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OPAL Server doesn't clean up symbolic links when github is down #634

OPAL Server doesn't clean up symbolic links when github is down #634

kreyyser commented Aug 6, 2024

gemanor commented Sep 18, 2024

algora-pbc bot commented Sep 18, 2024

sroopsai commented Sep 20, 2024

onyedikachi-david commented Sep 20, 2024 •

edited

Loading

gemanor commented Sep 21, 2024

kreyyser commented Sep 23, 2024

gemanor commented Sep 30, 2024

kreyyser commented Sep 30, 2024

OPAL Server doesn't clean up symbolic links when github is down #634

OPAL Server doesn't clean up symbolic links when github is down #634

Comments

kreyyser commented Aug 6, 2024

gemanor commented Sep 18, 2024

algora-pbc bot commented Sep 18, 2024

💎 $100 bounty • Permit.io

Steps to solve:

sroopsai commented Sep 20, 2024

onyedikachi-david commented Sep 20, 2024 • edited Loading

gemanor commented Sep 21, 2024

kreyyser commented Sep 23, 2024

gemanor commented Sep 30, 2024

kreyyser commented Sep 30, 2024

onyedikachi-david commented Sep 20, 2024 •

edited

Loading