Allow optional end-date in get_execution_stats()

[IMC07]

Description

Add optional parameter of max end date for TaskDependcySensor get_execution_stats(). This is a minor change in order to not introduce a breaking change. However, if you agree I think it is more beautiful to have: 1) a function that returns the latest state of the run to be used for the 'poking' sensor, and have another 2) functionality that allows to query the API in a more flexible manner and allows to return more from the json response.
This way we are more flexible, so we can use the start/end/state details to trigger different paths in the Brickflow DAGs (with IfElseConditionTask tasks) for instance. E.g. if the Airflow dependency was finished before time x, then branch in this direction, otherwise y direction. Or if latest run had status 'failed', then branch in direction z etc.

Right now it could be used like this:

tds = TaskDependencySensor(
            task_id=job["dag_id"],
            airflow_auth=AirflowProxyOktaClusterAuth(
                oauth2_conn_id=oauth2_conn_id,
                airflow_cluster_url=job["cluster_url"],
                airflow_version="2.0.2",
            ),
            external_dag_id=job["dag_id"],
            external_task_id=job["task_id"],
            allowed_states=["success"]
        )

execution_date = parse("2024-09-10T04:00:00.000000+00:00")
task_status = tds.get_execution_stats(execution_date=execution_date, max_end_date=max_end_date)

Related Issue

#154

Motivation and Context

See issue/above.

How Has This Been Tested?

Tested locally and with unit tests.

Screenshots (if appropriate):

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have read the CONTRIBUTING document.
• I have added tests to cover my changes.
• All new and existing tests passed.

[asingamaneni]

@IMC07 Looks like the format needs to be changed for the files

[IMC07]

@asingamaneni I forgot to run black, done now. Please check again.

[asingamaneni]

LGTM

[asingamaneni]

Can you try to run make cov in your local and test it once again

[asingamaneni]

Thanks for your contribution