Undersampling project proposal #34

kindalime · 2021-06-14T18:54:12Z

Proposal for the undersampling project discussed in #33.

Create directory for project proposals

Creating Sklearn proposal

Sklearn Example

* copies the sklearn example from TFX source and updates it to be run from a cloned tfx-addons repo instead

google-cla · 2021-06-14T18:54:17Z

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

What to do if you already signed the CLA

Individual signers

It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

google-cla · 2021-06-14T23:16:09Z

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

What to do if you already signed the CLA

Individual signers

It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

google-cla · 2021-06-14T23:20:01Z

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

What to do if you already signed the CLA

Individual signers

It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

google-cla · 2021-06-14T23:20:13Z

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

What to do if you already signed the CLA

Individual signers

It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

kindalime · 2021-06-14T23:51:00Z

@googlebot I signed it!

rcrowe-google

This looks good overall! I would discourage the idea of using BigQuery for this, since that would limit it to use on the Google Cloud only.

rcrowe-google · 2021-06-18T19:23:28Z

There is a related discussion and implementation in the TFX repo.

kindalime · 2021-06-29T14:23:07Z

Hello! Just a quick update: I've completed an initial version of the undersampling component, which can be found here. Constructive criticism is much appreciated!

zhitaoli · 2021-07-02T16:16:24Z

proposals/undersampling_project_proposal.md

+
+## Project Description
+
+This project will be a custom function-based component that inputs an artifact in `tfRecord` format of `tf.Example`s and randomly undersamples it, reducing the data to the lowest-frequency class. It will primarily use an underlying Apache Beam pipeline that will be wrapped inside the TensorFlow component.


(non blocking)

How much do you imagine this implementation will be tied to either the format or the tf.Example payload? If it becomes a different container, or a different payload, would sample still work?

While this implementation is tied to the tf.Example payload, this is the standard pipeline data flow for pipelines that are downstream from ExampleGen. We are planning for data format logic to be handled by ExampleGen; as long as your ExampleGen component or a custom component can ingest a requested data format, the Undersampler can use it.

zhitaoli · 2021-07-02T16:17:47Z

proposals/undersampling_project_proposal.md

+
+## Project Implementation
+
+At a high level, the plan is to use Apache Beam to injest a `tfRecord` of `tf.Examples`, shuffle them, convert them into a key-value `PCollection` with keys as class values and values as data points, and then use `Sample.FixedSizePerKey()` in order to perform the actual undersampling. The algorithm will be written as an Apache Beam pipeline, which will be wrapped into a TensorFlow custom function component to use with TFX pipelines. The component would be written as inputting a `tfRecord` artifact and exporting a similar `tf.Record` artifact, making its placement in a pipeline nearly ubiquitous. If necessary, the component may also be changed to a fully-custom component, albeit one where only the executor is edited.


What's the use case of TensorFlow custom function, vs just another Python function?

This is actually outdated, but the use of a TFX custom component function would be to allow its use in TFX pipelines as a component. The current implementation uses a fully-custom component with a custom executor and spec, which fulfills a similar role while allowing more complexity.

kindalime · 2021-07-06T17:52:52Z

Just a quick update: a finished initial version of the component can be found here.

pull_request_template.md

rcrowe-google · 2021-07-16T23:08:28Z

Closed accidentally

mihaimaruseac and others added 23 commits February 5, 2021 11:12

Initial

106996d

Add empty README

321f709

Adding README

fa03be8

Small change to README

5baaf61

Changing Baris and Hamza to ZenML

e91574c

Adding Sklearn example proposal

7152b68

Create directory for project proposals

bab96a4

Merge pull request tensorflow#18 from tensorflow/theadactyl-patch-1

ce55f63

Create directory for project proposals

Rearranging the files a bit

55b01b7

Typo

052f61c

Add Sklearn proposal

a0d9b21

Move Sklearn proposal

6f67dd8

Merge pull request tensorflow#19 from rcrowe-google/main

7e0407e

Creating Sklearn proposal

Small edit

5ada1c9

Merge pull request tensorflow#20 from rcrowe-google/main

2c6fbd5

Sklearn Example

More detail on the idea to proposal flow

337eae4

Fix link

a70a0c2

Setting up Sklearn example

16c75e9

Init sklearn readme

14766a4

add sklearn penguins example (tensorflow#27)

4dc9659

* copies the sklearn example from TFX source and updates it to be run from a cloned tfx-addons repo instead

Adding core team to approve project proposals

c34855d

Removing core team, since there are organization issues

6a7d222

Slight changes in approval process

8e33c66

kindalime changed the title ~~Add undersampling project proposal~~ Undersampling project proposal Jun 14, 2021

Add undersampling project proposal

c8532e0

kindalime force-pushed the main branch from fb9a636 to c8532e0 Compare June 14, 2021 23:16

rcrowe-google reviewed Jun 15, 2021

View reviewed changes

rcrowe-google mentioned this pull request Jun 18, 2021

[Discussion] Random Under-sampling for Imbalanced Datasets tensorflow/tfx#3831

Closed

Create pull_request_template.md

711a2ed

zhitaoli reviewed Jul 2, 2021

View reviewed changes

kindalime mentioned this pull request Jul 6, 2021

Undersampler Component Initial Look #37

Closed

Update project proposal

c3d03ee

casassg approved these changes Jul 7, 2021

View reviewed changes

pull_request_template.md Outdated Show resolved Hide resolved

Delete pull request template

dbdb865

kindalime marked this pull request as ready for review July 7, 2021 18:13

kindalime requested review from mihaimaruseac and theadactyl as code owners July 7, 2021 18:13

rcrowe-google closed this Jul 16, 2021

rcrowe-google force-pushed the main branch from 5829673 to 5ac13be Compare July 16, 2021 22:32

kindalime mentioned this pull request Jul 21, 2021

Sampling project proposal #47

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Undersampling project proposal #34

Undersampling project proposal #34

kindalime commented Jun 14, 2021

google-cla bot commented Jun 14, 2021

google-cla bot commented Jun 14, 2021

google-cla bot commented Jun 14, 2021

google-cla bot commented Jun 14, 2021

kindalime commented Jun 14, 2021

rcrowe-google left a comment

rcrowe-google commented Jun 18, 2021

kindalime commented Jun 29, 2021

zhitaoli Jul 2, 2021

kindalime Jul 6, 2021

zhitaoli Jul 2, 2021

zhitaoli Jul 2, 2021

kindalime Jul 6, 2021

kindalime commented Jul 6, 2021

rcrowe-google commented Jul 16, 2021


		## Project Description

		This project will be a custom function-based component that inputs an artifact in `tfRecord` format of `tf.Example`s and randomly undersamples it, reducing the data to the lowest-frequency class. It will primarily use an underlying Apache Beam pipeline that will be wrapped inside the TensorFlow component.


		## Project Implementation

		At a high level, the plan is to use Apache Beam to injest a `tfRecord` of `tf.Examples`, shuffle them, convert them into a key-value `PCollection` with keys as class values and values as data points, and then use `Sample.FixedSizePerKey()` in order to perform the actual undersampling. The algorithm will be written as an Apache Beam pipeline, which will be wrapped into a TensorFlow custom function component to use with TFX pipelines. The component would be written as inputting a `tfRecord` artifact and exporting a similar `tf.Record` artifact, making its placement in a pipeline nearly ubiquitous. If necessary, the component may also be changed to a fully-custom component, albeit one where only the executor is edited.

Undersampling project proposal #34

Undersampling project proposal #34

Conversation

kindalime commented Jun 14, 2021

google-cla bot commented Jun 14, 2021

What to do if you already signed the CLA

Individual signers

Corporate signers

google-cla bot commented Jun 14, 2021

What to do if you already signed the CLA

Individual signers

Corporate signers

google-cla bot commented Jun 14, 2021

What to do if you already signed the CLA

Individual signers

Corporate signers

google-cla bot commented Jun 14, 2021

What to do if you already signed the CLA

Individual signers

Corporate signers

kindalime commented Jun 14, 2021

rcrowe-google left a comment

Choose a reason for hiding this comment

rcrowe-google commented Jun 18, 2021

kindalime commented Jun 29, 2021

zhitaoli Jul 2, 2021

Choose a reason for hiding this comment

kindalime Jul 6, 2021

Choose a reason for hiding this comment

zhitaoli Jul 2, 2021

Choose a reason for hiding this comment

zhitaoli Jul 2, 2021

Choose a reason for hiding this comment

kindalime Jul 6, 2021

Choose a reason for hiding this comment

kindalime commented Jul 6, 2021

rcrowe-google commented Jul 16, 2021