Consider adding validators package

[sobolevn]

Currently a lot of python tools just throw exceptions at us or even uses: .is_valid() and .errors properties. It is not type-safe at all. Sometimes it is hard to explain to people how to really use type-driven design and make invalid states unreachable.

My idea is that we need to write thin wrappers around popular typed validation packages: like pydantic, cerebrus, schema, etc to return really typed things.

We also need to think about API to be expressive and recognisable enough.

Related: https://gist.github.com/maksbotan/14b4bebda2acab98cdd158f85a970855
CC @maksbotan

[sobolevn]

Shame on me! Not validators but parsers!

See https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/

[stereobutter]

there is also https://validators.readthedocs.io/en/latest/ which imho fortunately returns errors instead of raising them

[sobolevn]

1. https://hackage.haskell.org/package/monad-validate-1.2.0.0/docs/Control-Monad-Validate.html
2. https://github.com/debasishg/hask/blob/master/hfrdomain/README.md

[sobolevn]

Or we can add validate function to the pipeline.py.
Probably in the next release.

[sobolevn]

https://dry-rb.org/gems/dry-monads/1.3/validated/

[sobolevn]

https://typelevel.org/cats/datatypes/validated.html

[sobolevn]

So, it looks like Validated should be defined like so:

class Validated(Generic[_ValueType, _ErrorType]):
    _inner_value: Iterable[
        Callable[[_ValueType], Result[_ValueType, _ErrorType]],
    ]

It should work with all types: Maybe, FutureResult, IOResult, and Result, somehow. Union?

[sobolevn]

https://typelevel.org/cats/datatypes/validated.html#validated-vs-either

[maksbotan]

Take a look at http://hackage.haskell.org/package/these as well.

http://hackage.haskell.org/package/these-1.1/docs/Data-These.html#t:These
http://hackage.haskell.org/package/monad-chronicle-1/docs/Control-Monad-Chronicle.html

It's Monad instance has the same problems as Validated's, of course.

[sobolevn]

I am using These in fp-ts, it is quite useful when working with components in vue.
But, I don't know how one can use it in Python? Can you please share some usecases?

[astynax]

Here is another nice lib for validation: https://github.com/typeable/validationt
This one collects warnings but stops on errors - a pretty convenient behavior IMHO. Also you can get more than one warning for each field - it's nice too.

[sobolevn]

https://habr.com/ru/post/429104/

[sobolevn]

Current validation prototype:

from returns.primitives.hkt import KindN, kinded, Kinded
from returns.interfaces.specific.result import ResultLikeN
from returns.primitives.container import BaseContainer
from typing import TypeVar, Sequence, Callable, Iterable, TYPE_CHECKING
from returns._generated.iterable import iterable_kind

# Definitions:
_FirstType = TypeVar('_FirstType')
_SecondType = TypeVar('_SecondType')
_ThirdType = TypeVar('_ThirdType')

_ResultKind = TypeVar('_ResultKind', bound=ResultLikeN)

if not TYPE_CHECKING:
    reveal_type = print



# TODO:
def validate(
    items: Iterable[
        Callable[
            [_FirstType],
            KindN[_ResultKind, _FirstType, _SecondType, _ThirdType],
        ]
    ],
) -> Kinded[Callable[
    [_FirstType],
    KindN[_ResultKind, _FirstType, Sequence[_SecondType], _ThirdType],
]]:
    @kinded
    def factory(instance: _FirstType) -> KindN[
        _ResultKind,
        _FirstType,
        Sequence[_SecondType],
        _ThirdType,
    ]:
        swapped = [item.swap() for item in items]
        return iterable_kind(type(swapped[0]), swapped).swap()
    return factory

What can be changed / improved?

1. We need to add Better helpers for working with collections #526 support
2. We also need to change the function's signature to be validate(items, *, strategy: _ValidationStrategy = ...). Where _ValidationStrategy is some kind of a callable protocol. There can be multiple strategies, including: collecting all erorrs for a field vs collecting only a single (first / last) error for each field. So, the code would change like so: returns strategy(swapped).swap()
3. We also need to provide a higher-level abstraction to create validation pipelines. Because, currently there's no way to define that in a reasonable manner. Why do we need this? Because each validation item must have similar KindN[_ResultKind, ...] types. So, we cannot combine IOResult and Result validations in a single call:

import attr
from returns.result import Result
from returns.io import IOResult
from returns.context import ReaderResult

@attr.dataclass
class Person(object):
    fullname: str
    age: int
    passport: str

def validate_fullname(person: Person) -> Result[Person, str]:
    if not person.fullname:
        return Result.from_failure('No fullname specified')
    return Result.from_value(person)

def validate_age(person: Person) -> Result[Person, str]:
    if person.age < 0:
        return Result.from_failure('Negative age')
    return Result.from_value(person)

def validate_passport(person: Person) -> IOResult[Person, str]:
    """Impures, calls 3rd party API."""
    if not person.passport:  # this is not an IO action, just an example
        return IOResult.from_failure('Missing passort')
    return IOResult.from_value(person)

def validate_with_context(person: Person) -> ReaderResult[Person, str, int]:
    """Requires ``int`` context to actually validate anything."""
    def factory(deps: int) -> Result[Person, str]:
        if person.age < deps:
            return Result.from_failure('Less than minimal {0} age'.format(deps))
        return Result.from_value(person)
    return ReaderResult(factory)

person = Person('', 28, '')

simple = validate([
    validate_fullname,
    validate_age,
])(person)

hard = validate([
    validate_passport,
])(person)

context = validate([
    validate_with_context
])(person)

reveal_type(simple)
reveal_type(hard)
reveal_type(context(35))

Outputs:

experiments/validate.py:94: note: Revealed type is 'returns.result.Result*[validate.Person*, typing.Sequence[builtins.str*]]'
experiments/validate.py:95: note: Revealed type is 'returns.io.IOResult*[validate.Person*, typing.Sequence[builtins.str*]]'
experiments/validate.py:96: note: Revealed type is 'returns.result.Result[validate.Person*, typing.Sequence*[builtins.str*]]'

So, we need to somehow turn simple, hard, and context into a single call.

[sobolevn]

We can also add simple types to make validation more meaningful:

from returns.validation import Valid, Invalid, Validated

def register_user(user: Valid[User]) -> None:
     api.register_user_call(user.value)

Where Validated = Union[Valid[_ValueType], Invalid[_ValueType]]

[sobolevn]

https://kowainik.github.io/posts/haskell-mini-patterns#phantom-type-parameters

[sobolevn]

Related article: https://blog.drewolson.org/declarative-validation

PureScript: https://pursuit.purescript.org/packages/purescript-validation/5.0.0
Kotlin: https://arrow-kt.io/docs/apidocs/arrow-core/arrow.core/-validated/