docs: [FC-0074] add docs about creating new filters with pipeline steps #242

mariajgrimaldi · 2024-12-25T14:34:49Z

Description

Update how-to documents to create and implement pipeline steps considering design suggestions introduced in #240.

Here are the main documents to review:

How to Create a New Filter with Long-Term Support: https://github.com/openedx/openedx-filters/pull/242/files#diff-2825244e325626bcad68d204d59aa3e6ff696d1b5043fd4b990bd77e88dbb9a1
Implement Pipeline Steps: https://github.com/openedx/openedx-filters/pull/242/files#diff-f1e167403e1377f568c3a2641fb3a151cd61d26887b3a468f91669d4494aee61

Testing instructions

You can review the docs here: https://docsopenedxorg--242.org.readthedocs.build/projects/openedx-filters/en/242/how-tos/index.html

Deadline

None

Checklists

Check off if complete or not applicable:

Merge Checklist:

Post Merge:

Create a tag
Create a release on GitHub
Check new version is pushed to PyPI after tag-triggered build is
finished.
Delete working branch (if not needed anymore)
Upgrade the package in the Open edX platform requirements (if applicable)

openedx-webhooks · 2024-12-25T14:34:53Z

Thanks for the pull request, @mariajgrimaldi!

This repository is currently maintained by @openedx/hooks-extension-framework.

Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review.

🔘 Get product approval

If you haven't already, check this list to see if your contribution needs to go through the product review process.

If it does, you'll need to submit a product proposal for your contribution, and have it reviewed by the Product Working Group.
- This process (including the steps you'll need to take) is documented here.
If it doesn't, simply proceed with the next step.

🔘 Provide context

To help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:

Dependencies

This PR must be merged before / after / at the same time as ...
Blockers

This PR is waiting for OEP-1234 to be accepted.
Timeline information

This PR must be merged by XX date because ...
Partner information

This is for a course on edx.org.
Supporting documentation
Relevant Open edX discussion forum threads

🔘 Get a green build

If one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green.

Where can I find more information?

If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:

When can I expect my changes to be merged?

Our goal is to get community contributions seen and reviewed as efficiently as possible.

However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:

The size and impact of the changes that it introduces
The need for product review
Maintenance status of the parent repository

💡 As a result it may take up to several weeks or months to complete a review and merge your PR.

sarina · 2025-01-06T12:51:00Z

docs/how-tos/create-a-pipeline-step.rst

+
+This will mainly make the filters available for your CI/CD pipeline and local development environment. If you are using the Open edX platform, the library should be already be installed in the environment so no need to install it.
+
+Step 4: Create a Pipeline Step


This goes from step 2 to step 4!

I totally missed this. Thank you!

sarina · 2025-01-06T12:52:54Z

docs/how-tos/create-a-pipeline-step.rst

+- Limit each step to a single responsibility to make the code easier to maintain and test.
+- Keep the pipeline step logic simple and focused on the specific task it needs to perform.
+- Consider the performance implications of the pipeline step and avoid adding unnecessary complexity or overhead, considering the pipeline will be executed each time the filter is triggered.
+- Implement error handling and logging in the pipeline step to handle exceptions and provide useful information for debugging, considering both development and production environments.


Does it make sense to include an example of logging or verbose error messaging in your simple example above?

I added some. This suggestion mainly focuses on helping developers understand when the filter is doing what it's supposed to do based on the given conditions. Do you think the example illustrates that?

docs/how-tos/create-a-pipeline-step.rst

sarina · 2025-01-06T12:55:32Z

docs/reference/architecture-subdomains.rst

+-------------------+----------------------------------------------------------------------------------------------------+
+| Learning          | Allows learners to consume content and perform actions in a learning activity on the platform.     |
+-------------------+----------------------------------------------------------------------------------------------------+
+| Analytics         | Provides insights into learner behavior and course performance.                                    |


Suggested change

| Analytics | Provides insights into learner behavior and course performance. |

| Analytics | A look into learner behavior and course performance. |

I don't love this but I think you should avoid using "insights" to not confuse with Insights, the deprecated analytics product.

I used "visbility" instead of insights, what do you think?

MaferMazu

I have one suggestion, but the rest looks good to me.

Thanks @mariajgrimaldi ✨

MaferMazu · 2025-01-07T21:47:44Z

docs/how-tos/create-a-pipeline-step.rst

+- You understand the concept of filters or have reviewed the relevant :doc:`/concepts/index` docs.
+- You are familiar with the terminology used in the project, such as the terms :term:`Filter Type`. If not, you can review the :doc:`../reference/glossary` docs.
+- You have reviewed the :doc:`../decisions/0007-filter-design-practices` ADR.
+- You have identified that you need to create a new filter and have a use case for the filter.


My only comment is that we may want to drop the last assumption because, in this doc, we are talking about pipelines, and you cover the understanding part in Step 1.

Thanks for the suggestion! I replaced this line with understanding the pipeline step logic to implement instead: 667abe5

MaferMazu · 2025-01-07T23:28:44Z

docs/how-tos/create-a-pipeline-step.rst

+
+Before creating a pipeline step, you should understand your use case for the filter and the specific logic you want to implement in the pipeline step. In our example, we want to prevent users from enrolling in a course if they do not have a valid email address. We will create a pipeline step that checks if the user's email address is valid and raise an exception if it is not.
+
+You should review the list of filters available in the Open edX platform and identify the filter that best fits your use case. In our example, we will use the `CourseEnrollmentStarted filter`_ to implement the logic for our use case. You should review the filter's arguments to understand the data that will be passed to the pipeline step and the expected output. This will help you define the pipeline step's logic and signature.


Suggested change

You should review the list of filters available in the Open edX platform and identify the filter that best fits your use case. In our example, we will use the `CourseEnrollmentStarted filter`_ to implement the logic for our use case. You should review the filter's arguments to understand the data that will be passed to the pipeline step and the expected output. This will help you define the pipeline step's logic and signature.

You should review the `list of filters`_ available in the Open edX platform and identify the filter that best fits your use case. In our example, we will use the `CourseEnrollmentStarted filter`_ to implement the logic for our use case. You should review the filter's arguments to understand the data that will be passed to the pipeline step and the expected output. This will help you define the pipeline step's logic and signature.

I added this suggestion because I saw something similar in the openedx-events PR, and it took me to Reference/Events. I didn't know that page existed. I think it should be helpful to be able to redirect the people to the list of filters here: https://docs.openedx.org/projects/openedx-filters/en/latest/reference/filters.html#

Note: In the suggestion, we need to reference the link because I didn't put it.

This is pretty useful, thank you! ad4a794

MaferMazu · 2025-01-09T15:26:28Z

Thanks for addressing my feedback. This looks good to me ✅

…filters

openedx-webhooks added the open-source-contribution PR author is not from Axim or 2U label Dec 25, 2024

mariajgrimaldi mentioned this pull request Dec 26, 2024

docs: [FC-0074] add suggestion about the design of filters #240

Merged

15 tasks

mariajgrimaldi changed the title ~~Mjg/create a new filter~~ docs: add docs about creating new filters with pipeline steps Dec 26, 2024

mariajgrimaldi changed the title ~~docs: add docs about creating new filters with pipeline steps~~ docs: [FC-0074] add docs about creating new filters with pipeline steps Dec 26, 2024

mariajgrimaldi added the FC Relates to an Axim Funded Contribution project label Dec 26, 2024

mariajgrimaldi marked this pull request as ready for review December 26, 2024 13:06

mariajgrimaldi requested a review from a team as a code owner December 26, 2024 13:06

mariajgrimaldi requested a review from sarina December 26, 2024 13:10

sarina approved these changes Jan 6, 2025

View reviewed changes

sarina approved these changes Jan 7, 2025

View reviewed changes

MaferMazu approved these changes Jan 7, 2025

View reviewed changes

MaferMazu reviewed Jan 7, 2025

View reviewed changes

mariajgrimaldi mentioned this pull request Jan 8, 2025

docs: [FC-0074] add note about debugging pipeline steps #247

Open

15 tasks

Base automatically changed from MJG/practices-suggestions-adr to main January 14, 2025 09:44

mariajgrimaldi added 14 commits January 14, 2025 10:46

docs: add create a new filter documents

f8775df

docs: drop using filter in favor of create a new filter

2333321

docs: update documentation with latest how-to steps

77a4669

fix: remove trailing whitespace

9e9afc4

docs: mention filter configuration and replace filter content with args

421ee35

refactor: move implement pipeline steps to step 7

89181e1

docs: clarify identify triggering logic

32bcfe1

docs: update concepts so it references how-tos index

57c8093

docs: add create pipeline step documents

1d6804a

docs: add new line at the end of index

58a459e

refactor: add missing references to documents

5ef0db2

docs: add note about implementing multiple exceptions

ba012b3

refactor: address PR reviews

4dedce2

refactor: address PR review about arch subdomains

4702650

mariajgrimaldi added 2 commits January 14, 2025 10:46

docs: reference understanding the pipeline step logic instead of the …

34d2665

…filters

docs: reference list of filters in docs

de53680

mariajgrimaldi force-pushed the MJG/create-a-new-filter branch from ad4a794 to de53680 Compare January 14, 2025 09:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

docs: [FC-0074] add docs about creating new filters with pipeline steps #242

docs: [FC-0074] add docs about creating new filters with pipeline steps #242

mariajgrimaldi commented Dec 25, 2024 •

edited

Loading

openedx-webhooks commented Dec 25, 2024 •

edited

Loading

🔘 Get product approval

🔘 Provide context

🔘 Get a green build

Where can I find more information?

When can I expect my changes to be merged?

sarina Jan 6, 2025

mariajgrimaldi Jan 7, 2025

sarina Jan 6, 2025

mariajgrimaldi Jan 7, 2025

sarina Jan 6, 2025

mariajgrimaldi Jan 7, 2025

MaferMazu left a comment

MaferMazu Jan 7, 2025

mariajgrimaldi Jan 8, 2025

MaferMazu Jan 7, 2025 •

edited

Loading

mariajgrimaldi Jan 8, 2025

MaferMazu commented Jan 9, 2025


		This will mainly make the filters available for your CI/CD pipeline and local development environment. If you are using the Open edX platform, the library should be already be installed in the environment so no need to install it.

		Step 4: Create a Pipeline Step

	\| Analytics \| Provides insights into learner behavior and course performance. \|
	\| Analytics \| A look into learner behavior and course performance. \|


		Before creating a pipeline step, you should understand your use case for the filter and the specific logic you want to implement in the pipeline step. In our example, we want to prevent users from enrolling in a course if they do not have a valid email address. We will create a pipeline step that checks if the user's email address is valid and raise an exception if it is not.

		You should review the list of filters available in the Open edX platform and identify the filter that best fits your use case. In our example, we will use the `CourseEnrollmentStarted filter`_ to implement the logic for our use case. You should review the filter's arguments to understand the data that will be passed to the pipeline step and the expected output. This will help you define the pipeline step's logic and signature.

docs: [FC-0074] add docs about creating new filters with pipeline steps #242

Are you sure you want to change the base?

docs: [FC-0074] add docs about creating new filters with pipeline steps #242

Conversation

mariajgrimaldi commented Dec 25, 2024 • edited Loading

Description

Testing instructions

Deadline

Checklists

openedx-webhooks commented Dec 25, 2024 • edited Loading

🔘 Get product approval

🔘 Provide context

🔘 Get a green build

Where can I find more information?

When can I expect my changes to be merged?

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MaferMazu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MaferMazu Jan 7, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MaferMazu commented Jan 9, 2025

mariajgrimaldi commented Dec 25, 2024 •

edited

Loading

openedx-webhooks commented Dec 25, 2024 •

edited

Loading

MaferMazu Jan 7, 2025 •

edited

Loading