Os Error: too many files open + Error sending Request

[emilvincentz]

The app has started to crash + not loading spaces or images in chat properly + bug report also not working.

[emilvincentz]

Might also be connected to this loading errror:

[emilvincentz]

maybe related to this?
error loading image in chat #1261

[gnunicorn]

looks like we are leaking some file descriptor(s) ... there is plenty of reasons why that could be and maybe we just want to "resolves" this problem for now by reacting on that particular error code with prompting the user to restart the app ... it is hard for us to investigate that at the moment ..

[emilvincentz]

This now happens all the time and doesn't help with restarting the app

[emilvincentz]

[gnunicorn]

@emilvincentz can you confirm somewhere that this is because of the 'too many files open'? Neither of these messages you posted shows that ...

[emilvincentz]

well the last one Im not 100% sure of. All the loading error just started appearing at ones, along with the crashes and fail to bug report too, so thought they might be all connected, but maybe not

[emilvincentz]

It seems that they somehow are related. At least the two main messages that keeps appearing is the "Error sending request" and "Too many files open".
Below you can see that error messages changes between these two error messages, compared to the ones above.
Also when trying to bug report and error message shows, the app freezes.

[emilvincentz]

On IOS these errors are now happening all the time

[gnunicorn]

On IOS these errors are now happening all the time

Is that on release or nightly? I'll investigate what is going on ...

[emilvincentz]

just tested, release on App Store works fine. Only on the latest couple of nightlies where this happens

[gnunicorn]

okay. thanks for the info. according to logs of https://github.com/acterglobal/a3-meta/issues/264 it already happens almost immediately after start/sync ... which is severely odd. I suspect the iOS spawn-process is doing something bad there that leads to a FO explosion...

[emilvincentz]

This still happens on latest nightly

[emilvincentz]

Still seeing this on latest release along with other loading issues

trim.86E22DA1-72B7-4B73-9ED2-00F8346702B0.MOV

[emilvincentz]

It also ended up freezing the app, so had to restart which resolved it

[gnunicorn]

Okay, I have another piece of evidence:

it appears that our internal http-client doesn't limit the number of concurrent requests. And our UI issues a bunch of load stuff (e.g. user & room avatars, user counts, space sync ups) when we start up. And combined with the server rate limits that means a bunch of them are held open for a bit before they are responded to. On my local linux install with my larger account, I can see upwards of 250 pending concurrent requests with lsof:

acter   441431  ben 363u     IPv4            5162956       0t0      TCP slimbook:37972->h3002795.stratoserver.net:https (ESTABLISHED)
acter   441431  ben 364u     IPv4            5165974       0t0      TCP slimbook:38736->h3002795.stratoserver.net:https (ESTABLISHED)

It's fair to assume the number growth linear to the number of rooms the user is in. Thus if the numbers are even have way similar iOS is probably shutting us off with the IO error you are seeing as iOS has lower limits on open files per app.

[gnunicorn]

expected to be fixed with #1898