Population strategies and settings (#30) 🪲

* Population strategies and pluggable parsers 🧝🏻‍♀️

* Fixed Python 3.10 tests

* Disable Python 3.10 tests

* Fixing code-style issues

* Coverage settings

* Docs

* Code-style changes

* Release date

.codecov.yml

@@ -0,0 +1,20 @@
+coverage:
+  precision: 2
+  round: down
+  range: "70...100"
+
+  status:
+    project: false
+    patch: true
+    changes: false
+
+comment:
+  layout: "header, diff, changes, tree"
+  behavior: default
+
+ignore:
+  - "docs/**"
+  - ".github/**"
+  - "tests/**"
+  - "setup.py"
+  - "runtests.py"

.flake8

@@ -1,4 +1,3 @@
 [flake8]
-ignore = E203,W503,E704
 exclude = .git,.mypy_cache,.pytest_cache,.tox,.venv,__pycache__,build,dist,docs,venv
 max-line-length = 119

.github/ISSUE_TEMPLATE/feature_request.md

@@ -8,7 +8,7 @@ assignees: ''
 ---
 
 **Is your feature request related to a problem? Please describe.**
-A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+A clear and concise description of what the problem is. Ex. I'm always frustrated when...
 
 **Describe the solution you'd like**
 A clear and concise description of what you want to happen.

.github/workflows/tests.yml

@@ -31,6 +31,11 @@ jobs:
       run: |
         poetry run coverage run runtests.py
         poetry run coverage xml
+    - name: Run codacy-coverage-reporter
+      uses: codacy/codacy-coverage-reporter-action@v1
+      with:
+        project-token: ${{ secrets.CODACY_PROJECT_TOKEN }}
+        coverage-reports: coverage.xml
     - uses: codecov/codecov-action@v1
       with:
         token: ${{ secrets.CODECOV_TOKEN }}

CHANGELOG.md

@@ -1,8 +1,17 @@
 # Changelog
 
+## 0.20.0 : 14.10.2021
+
+Anniversary release 🥳
+
+- **Feature**: Population strategies introduced
+- **Feature**: `fill` method is deprecated and replaced by `populate`
+- **Feature**: `Settings` object introduced (`form.settings`)
+- **Feature**: Pluggable content-type parsers using `DJANGO_API_FORMS_PARSERS` setting
+
 ## 0.19.1 : 17.09.2021
 
-- **Typing**: `mime` argument in `FileField` is suppose to be a `tuple`
+- **Typing**: `mime` argument in `FileField` is supposed to be a `tuple`
 
 ## 0.19.0 : 12.07.2021
 

README.md

@@ -3,14 +3,14 @@
 [![PyPI version](https://badge.fury.io/py/django-api-forms.svg)](https://badge.fury.io/py/django-api-forms)
 [![codecov](https://codecov.io/gh/Sibyx/django_api_forms/branch/master/graph/badge.svg)](https://codecov.io/gh/Sibyx/django_api_forms)
 
-[Django Forms](https://docs.djangoproject.com/en/3.2/topics/forms/) approach in processing of RESTful HTTP request
-payload (especially for content type like [JSON](https://www.json.org/) or [MessagePack](https://msgpack.org/))
+[Django Forms](https://docs.djangoproject.com/en/3.2/topics/forms/) approach in the processing of a RESTful HTTP
+request payload (especially for content type like [JSON](https://www.json.org/) or [MessagePack](https://msgpack.org/))
 without HTML front-end.
 
 ## Motivation
 
 The main idea was to create a simple and declarative way to specify the format of expecting requests with the ability
-to validate them. Firstly I tried to use [Django Forms](https://docs.djangoproject.com/en/3.0/topics/forms/) to
+to validate them. Firstly, I tried to use [Django Forms](https://docs.djangoproject.com/en/3.0/topics/forms/) to
 validate my API requests (I use pure Django in my APIs). I have encountered a problem with nesting my requests without
 a huge boilerplate. Also, the whole HTML thing was pretty useless in my RESTful APIs.
 
@@ -24,10 +24,10 @@ I wanted to:
 I wanted to keep:
 
 - friendly declarative Django syntax,
-([DeclarativeFieldsMetaclass](https://github.com/django/django/blob/master/django/forms/forms.py#L22) is beautiful),
-- [Validators](https://docs.djangoproject.com/en/3.1/ref/validators/),
-- [ValidationError](https://docs.djangoproject.com/en/3.1/ref/exceptions/#validationerror),
-- [Form fields](https://docs.djangoproject.com/en/3.1/ref/forms/fields/) (In the end, I had to "replace" some of them).
+([DeclarativeFieldsMetaclass](https://github.com/django/django/blob/master/django/forms/forms.py#L25) is beautiful),
+- [Validators](https://docs.djangoproject.com/en/3.2/ref/validators/),
+- [ValidationError](https://docs.djangoproject.com/en/3.2/ref/exceptions/#validationerror),
+- [Form fields](https://docs.djangoproject.com/en/3.2/ref/forms/fields/) (In the end, I had to "replace" some of them).
 
 So I have decided to create a simple Python package to cover all my expectations.
 
@@ -65,6 +65,29 @@ INSTALLED_APPS = (
 )
 ```
 
+You can change the default behavior of population strategies or parsers using these settings (listed with default
+values). Keep in mind, that dictionaries are not replaced by your settings they are merged with defaults.
+
+For more information about the parsers and the population strategies check the documentation.
+
+```python
+DJANGO_API_FORMS_POPULATION_STRATEGIES = {
+    'django_api_forms.fields.FormFieldList': 'django_api_forms.population_strategies.IgnoreStrategy',
+    'django_api_forms.fields.FileField': 'django_api_forms.population_strategies.IgnoreStrategy',
+    'django_api_forms.fields.ImageField': 'django_api_forms.population_strategies.IgnoreStrategy',
+    'django_api_forms.fields.FormField': 'django_api_forms.population_strategies.IgnoreStrategy',
+    'django.forms.models.ModelMultipleChoiceField': 'django_api_forms.population_strategies.IgnoreStrategy',
+    'django.forms.models.ModelChoiceField': 'django_api_forms.population_strategies.ModelChoiceFieldStrategy'
+}
+
+DJANGO_API_FORMS_DEFAULT_POPULATION_STRATEGY = 'django_api_forms.population_strategies.BaseStrategy'
+
+DJANGO_API_FORMS_PARSERS = {
+    'application/json': 'json.loads',
+    'application/x-msgpack': 'msgpack.loads'
+}
+```
+
 ## Example
 
 **Simple nested JSON request**

django_api_forms/fields.py

@@ -16,12 +16,6 @@
 DATA_URI_PATTERN = r"data:((?:\w+\/(?:(?!;).)+)?)((?:;[\w=]*[^;])*),(.+)"
 
 
-class IgnoreFillMixin:
-    @property
-    def ignore_fill(self) -> bool:
-        return True
-
-
 class BooleanField(Field):
     def to_python(self, value):
         if value in (True, 'True', 'true', '1', 1):
@@ -91,7 +85,7 @@ def to_python(self, value) -> typing.List:
         return result
 
 
-class FormField(Field, IgnoreFillMixin):
+class FormField(Field):
     def __init__(self, form: typing.Type, **kwargs):
         self._form = form
 
@@ -112,7 +106,7 @@ def to_python(self, value) -> typing.Union[typing.Dict, None]:
             raise RequestValidationError(form.errors)
 
 
-class FormFieldList(FormField, IgnoreFillMixin):
+class FormFieldList(FormField):
     def __init__(self, form: typing.Type, min_length=None, max_length=None, **kwargs):
         self._min_length = min_length
         self._max_length = max_length
@@ -220,7 +214,7 @@ def to_python(self, value) -> typing.Union[typing.Dict, typing.List]:
         return value
 
 
-class FileField(Field, IgnoreFillMixin):
+class FileField(Field):
     default_error_messages = {
         'max_length': _('Ensure this file has at most %(max)d bytes (it has %(length)d).'),
         'invalid_uri': _("The given URI is not a valid Data URI."),
@@ -261,7 +255,7 @@ def to_python(self, value: str) -> typing.Optional[File]:
         return file
 
 
-class ImageField(FileField, IgnoreFillMixin):
+class ImageField(FileField):
     default_error_messages = {
         'invalid_image': _("Upload a valid image. The file you uploaded was either not an image or a corrupted image.")
     }

django_api_forms/forms.py

@@ -1,29 +1,20 @@
 import copy
-import json
+import warnings
 from typing import Union, List
 
 from django.core.exceptions import NON_FIELD_ERRORS, ValidationError
-from django.forms import ModelChoiceField, ModelMultipleChoiceField, fields_for_model
+from django.forms import fields_for_model
 from django.forms.forms import DeclarativeFieldsMetaclass
 from django.forms.models import ModelFormOptions
 from django.utils.translation import gettext as _
 
 from .exceptions import RequestValidationError, UnsupportedMediaType, ApiFormException
-
-try:
-    import msgpack
-
-    is_msgpack_installed = True
-except ImportError:
-    is_msgpack_installed = False
-
-parsers_by_content_type = {'application/json': json.loads}
-if is_msgpack_installed:
-    parsers_by_content_type['application/x-msgpack'] = msgpack.loads
+from .settings import Settings
+from .utils import resolve_from_path
 
 
 class BaseForm(object):
-    def __init__(self, data=None, request=None):
+    def __init__(self, data=None, request=None, settings: Settings = None):
         if data is None:
             self._data = {}
         else:
@@ -33,6 +24,7 @@ def __init__(self, data=None, request=None):
         self._dirty = []
         self.cleaned_data = None
         self._request = request
+        self.settings = settings or Settings()
 
         if isinstance(data, dict):
             for key in data.keys():
@@ -64,6 +56,8 @@ def create_from_request(cls, request):
         if not request.body:
             return cls()
 
+        settings = Settings()
+
         all_attributes = request.META.get('CONTENT_TYPE', '').replace(' ', '').split(';')
         content_type = all_attributes.pop(0)
 
@@ -72,14 +66,13 @@ def create_from_request(cls, request):
             key, value = attribute.split('=')
             optional_attributes[key] = value
 
-        parser = parsers_by_content_type.get(content_type)
+        if content_type not in settings.PARSERS:
+            raise UnsupportedMediaType()
 
-        if parser:
-            data = parser(request.body)
-        else:
-            raise UnsupportedMediaType
+        parser = resolve_from_path(settings.PARSERS[content_type])
+        data = parser(request.body)
 
-        return cls(data, request)
+        return cls(data, request, settings)
 
     @property
     def dirty(self) -> List:
@@ -159,12 +152,12 @@ def clean(self):
         """
         return self.cleaned_data
 
-    def fill(self, obj, exclude: List[str] = None):
-        """
-        :param exclude:
-        :param obj:
-        :return:
+    def populate(self, obj, exclude: List[str] = None):
         """
+                :param exclude:
+                :param obj:
+                :return:
+                """
         if exclude is None:
             exclude = []
 
@@ -182,38 +175,31 @@ def fill(self, obj, exclude: List[str] = None):
 
             # Skip default behaviour if there is fill_ method available
             if hasattr(self, f"fill_{key}"):
+                warnings.warn(
+                    "Form.fill_ methods are deprecated and will be not supported in 1.0. Use Form.populate_ instead.",
+                    DeprecationWarning,
+                    stacklevel=2
+                )
                 setattr(obj, key, getattr(self, f"fill_{key}")(obj, self.cleaned_data[key]))
                 continue
 
-            # Skip if field is not fillable
-            if hasattr(field, 'ignore_fill') and field.ignore_fill:
-                continue
-
-            # ModelMultipleChoiceField is not fillable (yet)
-            if isinstance(field, ModelMultipleChoiceField):
-                continue
-
-            """
-            We need to changes key postfix if there is ModelChoiceField (because of _id etc.)
-            We always try to assign whole object instance, for example:
-            artis_id is normalized as Artist model, but it have to be assigned to artist model property
-            because artist_id in model has different type (for example int if your are using int primary keys)
-            If you are still confused (sorry), try to check docs
-            """
-            if isinstance(field, ModelChoiceField):
-                model_key = key
-                if field.to_field_name:
-                    postfix_to_remove = f"_{field.to_field_name}"
-                else:
-                    postfix_to_remove = "_id"
-                if key.endswith(postfix_to_remove):
-                    model_key = key[:-len(postfix_to_remove)]
-                setattr(obj, model_key, self.cleaned_data[key])
-            else:
-                setattr(obj, key, self.cleaned_data[key])
+            field_class = f"{field.__class__.__module__}.{field.__class__.__name__}"
+            strategy = resolve_from_path(
+                self.settings.POPULATION_STRATEGIES.get(
+                    field_class, "django_api_forms.population_strategies.BaseStrategy"
+                )
+            )
+            strategy()(field, obj, key, self.cleaned_data[key])
 
         return obj
 
+    def fill(self, obj, exclude: List[str] = None):
+        warnings.warn(
+            "Form.fill() method is deprecated and will be removed in 1.0. Use Form.populate() instead.",
+            DeprecationWarning
+        )
+        return self.populate(obj, exclude)
+
 
 class ModelForm(BaseForm, metaclass=DeclarativeFieldsMetaclass):
     """

django_api_forms/population_strategies.py

@@ -0,0 +1,28 @@
+class BaseStrategy:
+    def __call__(self, field, obj, key: str, value):
+        setattr(obj, key, value)
+
+
+class IgnoreStrategy(BaseStrategy):
+    def __call__(self, field, obj, key: str, value):
+        pass
+
+
+class ModelChoiceFieldStrategy(BaseStrategy):
+
+    """
+    We need to changes key postfix if there is ModelChoiceField (because of _id etc.)
+    We always try to assign whole object instance, for example:
+    artis_id is normalized as Artist model, but it have to be assigned to artist model property
+    because artist_id in model has different type (for example int if your are using int primary keys)
+    If you are still confused (sorry), try to check docs
+    """
+    def __call__(self, field, obj, key: str, value):
+        model_key = key
+        if field.to_field_name:
+            postfix_to_remove = f"_{field.to_field_name}"
+        else:
+            postfix_to_remove = "_id"
+        if key.endswith(postfix_to_remove):
+            model_key = key[:-len(postfix_to_remove)]
+        setattr(obj, model_key, value)

django_api_forms/settings.py

@@ -0,0 +1,38 @@
+from django.conf import settings
+
+DEFAULTS = {
+    'POPULATION_STRATEGIES': {
+        'django_api_forms.fields.FormFieldList': 'django_api_forms.population_strategies.IgnoreStrategy',
+        'django_api_forms.fields.FileField': 'django_api_forms.population_strategies.IgnoreStrategy',
+        'django_api_forms.fields.ImageField': 'django_api_forms.population_strategies.IgnoreStrategy',
+        'django_api_forms.fields.FormField': 'django_api_forms.population_strategies.IgnoreStrategy',
+        'django.forms.models.ModelMultipleChoiceField': 'django_api_forms.population_strategies.IgnoreStrategy',
+        'django.forms.models.ModelChoiceField': 'django_api_forms.population_strategies.ModelChoiceFieldStrategy'
+    },
+    'DEFAULT_POPULATION_STRATEGY': 'django_api_forms.population_strategies.BaseStrategy',
+    'PARSERS': {
+        'application/json': 'json.loads',
+        'application/x-msgpack': 'msgpack.loads'
+    }
+}
+
+
+class Settings:
+    def __getattr__(self, item):
+        if item not in DEFAULTS:
+            raise AttributeError(f"'{self.__class__.__name__}' object has no attribute '{item}'")
+
+        django_setting = f"DJANGO_API_FORMS_{item}"
+        default = DEFAULTS[item]
+
+        if hasattr(settings, django_setting):
+            customized_value = getattr(settings, django_setting)
+            if isinstance(default, dict):
+                value = {**default, **customized_value}
+            else:
+                value = customized_value
+        else:
+            value = default
+
+        setattr(self, item, value)
+        return value