Skip to content

Commit

Permalink
[CLN] Warn when chroma_server_nofile is higher than sys hard limit (#…
Browse files Browse the repository at this point in the history
…1409)

## Description of changes

*Summarize the changes made by this PR.*
 - Improvements & Bug fixes
	 - Fixes #1404

## Test plan
*How are these changes tested?*

- [x] Tests pass locally with `pytest` for python, `yarn test` for js
- [x] Local repro and fix

This is one contiguous terminal session, broken into parts so shell and
python syntax highlighting both look nice:

```sh
(.venv) 0 beggers % git checkout main
M	chromadb/cli/cli.py
Already on 'main'
Your branch is up to date with 'origin/main'.
(.venv) 0 beggers % export CHROMA_SERVER_NOFILE=9223372036854775808                                                      ~/chroma/chroma
(.venv) 0 beggers % python
```
```python
Python 3.11.6 (main, Oct  2 2023, 20:46:14) [Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import chromadb
>>> c = chromadb.Client()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/beggers/chroma/chroma/chromadb/__init__.py", line 197, in Client
    return ClientCreator(tenant=tenant, database=database, settings=settings)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/beggers/chroma/chroma/chromadb/api/client.py", line 138, in __init__
    super().__init__(settings=settings)
  File "/Users/beggers/chroma/chroma/chromadb/api/client.py", line 41, in __init__
    SharedSystemClient._create_system_if_not_exists(self._identifier, settings)
  File "/Users/beggers/chroma/chroma/chromadb/api/client.py", line 48, in _create_system_if_not_exists
    new_system = System(settings)
                 ^^^^^^^^^^^^^^^^
  File "/Users/beggers/chroma/chroma/chromadb/config.py", line 322, in __init__
    raise ValueError(
ValueError: chroma_server_nofile cannot be set to a value greater than the current hard limit of 9223372036854775807
>>> ^D
```
```sh
(.venv) 0 beggers % git checkout beggers/nofile-limit 
M	chromadb/cli/cli.py
Switched to branch 'beggers/nofile-limit'
Your branch is up to date with 'origin/beggers/nofile-limit'.
(.venv) 0 beggers % python 
```
```python
Python 3.11.6 (main, Oct  2 2023, 20:46:14) [Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import chromadb
>>> c = chromadb.Client()
WARNING:root:chroma_server_nofile cannot be set to a value greater than the current hard limit of 9223372036854775807
>>> 
```

## Documentation Changes
*Are all docstrings for user-facing APIs updated if required? Do we need
to make documentation changes in the [docs
repository](https://github.com/chroma-core/docs)?*

---------

Co-authored-by: Hammad Bashir <[email protected]>
  • Loading branch information
beggers and HammadB authored Nov 17, 2023
1 parent 18eef42 commit c64cdc9
Showing 1 changed file with 3 additions and 3 deletions.
6 changes: 3 additions & 3 deletions chromadb/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -319,11 +319,11 @@ def __init__(self, settings: Settings):
desired_soft = settings["chroma_server_nofile"]
# Validate
if desired_soft > curr_hard:
raise ValueError(
f"chroma_server_nofile cannot be set to a value greater than the current hard limit of {curr_hard}"
logging.warning(
f"chroma_server_nofile cannot be set to a value greater than the current hard limit of {curr_hard}. Keeping soft limit at {curr_soft}"
)
# Apply
if desired_soft > curr_soft:
elif desired_soft > curr_soft:
try:
resource.setrlimit(
resource.RLIMIT_NOFILE, (desired_soft, curr_hard)
Expand Down

0 comments on commit c64cdc9

Please sign in to comment.