README.MD

Map Operator Example

Example of how to use the map operator. In this example, the map operator transforms masks SSN numbers in a user record.

With the dataflow.yaml file in the current directory, run the following commands:

sdf run

The sample data file used to run this test ./sample-data/data.txt has the following records:

{"name": "Alice", "ssn": "222-333-5555"}
{"name": "Bob", "ssn": "444-666-7777"}

Produce the data to the users topic:

fluvio produce users -f ./sample-data/data.txt

Checkout the data in users topic:

fluvio consume users -Bd

Consume from users-public to see them transformed:

fluvio consume users-public -Bd

{"name":"Alice","ssn":"***-***-****"}
{"name":"Bob","ssn":"***-***-****"}

Note: the ssn values are masked by the regex command.

Display the stateful dataflow stats in the sdf runtime >> terminal:

show state map-service/mask-ssn/metrics

Key    Window  succeeded  failed
stats  *       2          0

Exit sdf terminal and clean-up. The --force flag removes the topics:

sdf clean --force