-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support custom SASL mechanisms #2256
base: master
Are you sure you want to change the base?
Commits on Aug 20, 2021
-
Support custom SASL mechanisms
There is some interest in supporting various SASL mechanisms not currently included in the library: * dpkp#2110 (DMS) * dpkp#2204 (SSPI) * dpkp#2232 (AWS_MSK_IAM) Adding these mechanisms in the core library may be undesirable due to: * Increased maintenance burden. * Unavailable testing environments. * Vendor specificity. This commit provides a quick prototype for a pluggable SASL system. --- **Example** To define a custom SASL mechanism a module must implement two methods: ```py def validate_config(conn): # Check configuration values, available libraries, etc. assert conn.config['vendor_specific_setting'] is not None, ( 'vendor_specific_setting required when sasl_mechanism=MY_SASL' ) def try_authenticate(conn, future): # Do authentication routine and return resolved Future with failed # or succeeded state. ``` And then the custom mechanism should be registered before initializing a KafkaAdminClient, KafkaConsumer, or KafkaProducer: ```py import kafka.sasl from kafka import KafkaProducer import my_sasl kafka.sasl.register_mechanism('MY_SASL', my_sasl) producer = KafkaProducer(sasl_mechanism='MY_SASL') ``` --- **Notes** **ABCs** This prototype does not implement an ABC for custom SASL mechanisms. Using an ABC would reduce a few of the explicit assertions involved with registering a mechanism and is a viable option. Due to differing feature sets between py2/py3 this option was not explored, but shouldn't be difficult. **Private Methods** This prototype relies on some methods that are currently marked as **private** in `BrokerConnection`. * `._can_send_recv` * `._lock` * `._recv_bytes_blocking` * `._send_bytes_blocking` A pluggable system would require stable interfaces for these actions. **Alternative Approach** If the module-scoped dict modification in `register_mechanism` feels too clunky maybe the addtional mechanisms can be specified via an argument when initializing one of the `Kafka*` classes?
Configuration menu - View commit details
-
Copy full SHA for 03c357f - Browse repository at this point
Copy the full SHA 03c357fView commit details
Commits on Mar 7, 2024
-
Configuration menu - View commit details
-
Copy full SHA for b95e46d - Browse repository at this point
Copy the full SHA b95e46dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 78c74c0 - Browse repository at this point
Copy the full SHA 78c74c0View commit details
Commits on Mar 8, 2024
-
Configuration menu - View commit details
-
Copy full SHA for e796019 - Browse repository at this point
Copy the full SHA e796019View commit details -
Reconfigure tests to complete in a more timely manner and skip some i…
…terations for Kafka 0.8.2 and Python 3.12 (dpkp#159) * skip failing tests for PyPy since they work locally * Reconfigure tests for PyPy and 3.12 * Skip partitioner tests in test_partitioner.py if 3.12 and 0.8.2 * Update test_partitioner.py * Update test_producer.py * Timeout tests after ten minutes * Set 0.8.2.2 to be experimental from hereon * Formally support PyPy 3.9
Configuration menu - View commit details
-
Copy full SHA for 38e159a - Browse repository at this point
Copy the full SHA 38e159aView commit details
Commits on Mar 9, 2024
-
Test Kafka 0.8.2.2 using Python 3.10 in the meantime (dpkp#161)
* Test Kafka 0.8.2.2 using Python 3.11 in the meantime * Override PYTHON_LATEST conditionally in python-package.yml * Update python-package.yml * add python annotation to kafka version test matrix * Update python-package.yml * try python 3.10
Configuration menu - View commit details
-
Copy full SHA for e762321 - Browse repository at this point
Copy the full SHA e762321View commit details -
Remove support for EOL'ed versions of Python (dpkp#160)
* Remove support for EOL'ed versions of Python * Update setup.py
Configuration menu - View commit details
-
Copy full SHA for 00750aa - Browse repository at this point
Copy the full SHA 00750aaView commit details -
Stop testing Python 3.13 in python-package.yml (dpkp#162)
Too many MRs to review... so little time.
Configuration menu - View commit details
-
Copy full SHA for 5bd1323 - Browse repository at this point
Copy the full SHA 5bd1323View commit details -
Avoid 100% CPU usage while socket is closed (dpkp#156)
After stop/start kafka service, kafka-python may use 100% CPU caused by busy-retry while the socket was closed. This fix the issue by unregister the socket if the fd is negative. Co-authored-by: Orange Kao <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for cda8f81 - Browse repository at this point
Copy the full SHA cda8f81View commit details -
Fix DescribeConfigsResponse_v1 config_source (dpkp#150)
Co-authored-by: Ryar Nyah <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for c02df08 - Browse repository at this point
Copy the full SHA c02df08View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7f262e0 - Browse repository at this point
Copy the full SHA 7f262e0View commit details
Commits on Mar 10, 2024
-
Fix base class of DescribeClientQuotasResponse_v0 (dpkp#144)
Co-authored-by: Denis Otkidach <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 65eacfb - Browse repository at this point
Copy the full SHA 65eacfbView commit details -
Update license_file to license_files (dpkp#131)
The former has been deprecated since setuptools 56 Co-authored-by: micwoj92 <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for e0ebe5d - Browse repository at this point
Copy the full SHA e0ebe5dView commit details -
Update some RST documentation syntax (dpkp#130)
* docs: Update syntax in README.rst * docs: Update code block syntax in docs/index.rst --------- Co-authored-by: HalfSweet <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 26bb3eb - Browse repository at this point
Copy the full SHA 26bb3ebView commit details -
Fix crc32c's __main__ for Python 3 (dpkp#142)
* Fix crc32c's __main__ for Python 3 * Remove TODO from _crc32c.py --------- Co-authored-by: Yonatan Goldschmidt <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 88763da - Browse repository at this point
Copy the full SHA 88763daView commit details -
Strip trailing dot off hostname. (dpkp#133)
Co-authored-by: Dave Voutila <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for b1a4c53 - Browse repository at this point
Copy the full SHA b1a4c53View commit details -
Handle OSError to properly recycle SSL connection, fix infinite loop (d…
…pkp#155) * handling OSError * better error output * removed traceback logging --------- Co-authored-by: Alexander Sibiryakov <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 18eaa2d - Browse repository at this point
Copy the full SHA 18eaa2dView commit details -
client_async: Allow throwing an exception upon socket error during (d…
…pkp#134) wakeup When wakeup() is called, we sometime notice that we get an endless prints: "Unable to send to wakeup socket!". Those prints are spamming the logs. This commit aims to address it by allowing restating the application via an intentional exception raise. This behavior is configurable and its default is backward compatible. Signed-off-by: shimon-armis <[email protected]> Co-authored-by: shimon-armis <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 54cbd63 - Browse repository at this point
Copy the full SHA 54cbd63View commit details -
Configuration menu - View commit details
-
Copy full SHA for 82fae7a - Browse repository at this point
Copy the full SHA 82fae7aView commit details
Commits on Mar 12, 2024
-
Log connection errors at ERROR level (dpkp#139)
Co-authored-by: drewdogg <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for eb6fd9b - Browse repository at this point
Copy the full SHA eb6fd9bView commit details
Commits on Mar 17, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 035af5b - Browse repository at this point
Copy the full SHA 035af5bView commit details -
Configuration menu - View commit details
-
Copy full SHA for ebcfcb1 - Browse repository at this point
Copy the full SHA ebcfcb1View commit details