Skip to content

Learn how to develop and test stateful streaming and batch data pipelines

License

Notifications You must be signed in to change notification settings

mkuthan/stream-processing

This branch is up to date with main.

Folders and files

NameName
Last commit message
Last commit date

Latest commit

97d6adf · Jan 3, 2025
Jan 3, 2025
Jan 3, 2025
Nov 25, 2024
Nov 23, 2023
Feb 6, 2024
Aug 29, 2024
Aug 29, 2024
Dec 2, 2024
Oct 9, 2023
Jul 17, 2024
Oct 2, 2024
Oct 22, 2023
Jan 3, 2025
Sep 23, 2023
Oct 2, 2024
Jan 23, 2022
Oct 23, 2023
Oct 24, 2023

Repository files navigation

Learn How To Develop And Test Stateful Streaming Data Pipelines

CI codecov

Shared modules:

  • stream-processing-shared - shared utilities for developing stateful streaming data pipelines
  • stream-processing-infrastructure - infrastructure layer with IOs for BigQuery, Pubsub and Cloud Storage
  • stream-processing-test - shared utilities for testing stateful streaming data pipelines

Use cases:

  • toll-application, toll-domain, toll-infrastructure - sample application for toll data processing, see blog post
  • word-count - fixed window example, see blog post
  • session-window - session window example, see blog post