Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(etls): add interreg 2014UK16RFOP001 - EUBFR-258 #220

Open
wants to merge 12 commits into
base: master
Choose a base branch
from
1 change: 1 addition & 0 deletions config.example.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
"2014tc16rfcb047",
"2014tc16rfpc001",
"2014tc16rftn002",
"2014uk16rfop001",
"bulgaria",
"cordis",
"devco",
Expand Down
3 changes: 3 additions & 0 deletions docs/types/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,9 @@ Here's a list of the transformations made in ETLs around the `Project` model.
- [2014tc16rfcb047 - XLS](./etls/2014tc16rfcb047-xls.md)
- [2014tc16rfpc001 - XLS](./etls/2014tc16rfpc001-xls.md)
- [2014tc16rftn002 - XLS](./etls/2014tc16rftn002-xls.md)
- [2014uk16rfop001 - CSV](./etls/2014uk16rfop001-csv.md)
- [2014uk16rfop001 - ODS](./etls/2014uk16rfop001-ods.md)
- [2014uk16rfop001 - XLS](./etls/2014uk16rfop001-xls.md)
- [bulgaria - XLS](./etls/bulgaria-xls.md)
- [CORDIS - CSV](./etls/cordis-csv.md)
- [DEVCO - XLS](./etls/devco-xls.md)
Expand Down
162 changes: 162 additions & 0 deletions docs/types/etls/2014uk16rfop001-csv.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
<!-- Generated by documentation.js. Update this documentation by updating the source code. -->

## 2014uk16rfop001CsvTransform

Map fields for 2014uk16rfop001 producer, CSV file types

Example input data: [stub][1]

Transform function: [implementation details][2]

### Parameters

- `record` **[Object][3]** Piece of data to transform before going to harmonized storage.

Returns **Project** JSON matching the type fields.

### getBudget

Preprocess `budget`.

Input fields taken from the `record` are:

- `Total project costs �m (eligible project costs only)`
- `% of project funded by EU (Co-financing rate%)`

#### Parameters

- `record` **[Object][3]** The row received from parsed file

Returns **Budget**

### getDescription

Preprocess `description`.

Input fields taken from the `record` are:

- `Summary of project(max 100 words)`
- `Local enterprise partnership area`

#### Parameters

- `record` **[Object][3]** The row received from parsed file

Returns **[String][4]**

### getProjectId

Generates an ID for `project_id`.

Input fields taken from the `record` are:

- `Name of project`

#### Parameters

- `record` **[Object][3]** The row received from parsed file

Returns **[String][4]**

### getLocations

Preprocess `project_locations`.

Input fields taken from the `record` are:

- `Location (postcode)`

#### Parameters

- `record` **[Object][3]** The row received from parsed file

Returns **[Array][5]&lt;[Location][6]>**

### getThemes

Preprocess `themes`.

Input fields taken from the `record` are:

- `Type and focus support (category of intervention)`

#### Parameters

- `record` **[Object][3]** The row received from parsed file

Returns **[Array][5]&lt;[String][4]>**

### getThirdParties

Preprocess `third_parties`.

Input fields taken from the `record` are:

- `Recipient of funds(ERDF/ESF beneficiary)`

#### Parameters

- `record` **[Object][3]** The row received from parsed file

Returns **[Array][5]&lt;ThirdParty>**

### formatDate

Format date.

#### Parameters

- `date` **[Date][7]**

Returns **[Date][7]** The date formatted into an ISO 8601 date format

### getTimeframe

Preprocess `timeframe`.

Input fields taken from the `record` are:

- `Start date`
- `End date`

#### Parameters

- `record` **[Object][3]** The row received from parsed file

Returns **Timeframe**

### getTitle

Preprocess `title`.

Input fields taken from the `record` are:

- `Name of project`

#### Parameters

- `record` **[Object][3]** The row received from parsed file

Returns **[String][4]**

### getType

Preprocess `type`.

Input fields taken from the `record` are:

- `Type of fund`

#### Parameters

- `record` **[Object][3]** The row received from parsed file

Returns **[Array][5]&lt;[String][4]>**

[1]: https://github.com/ec-europa/eubfr-data-lake/blob/master/services/ingestion/etl/2014uk16rfop001/csv/test/stubs/record.json
[2]: https://github.com/ec-europa/eubfr-data-lake/blob/master/services/ingestion/etl/2014uk16rfop001/csv/src/lib/transform.js
[3]: https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Object
[4]: https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/String
[5]: https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Array
[6]: https://developer.mozilla.org/docs/Web/API/Location
[7]: https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Date
109 changes: 109 additions & 0 deletions docs/types/etls/2014uk16rfop001-ods.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
<!-- Generated by documentation.js. Update this documentation by updating the source code. -->

## 2014uk16rfop001XlsTransform

Map fields for 2014uk16rfop001 producer, XLS file types

Example input data: [stub][1]

Transform function: [implementation details][2]

### Parameters

- `record` **[Object][3]** Piece of data to transform before going to harmonized storage.

Returns **Project** JSON matching the type fields.

### getBudget

Preprocess `budget`.

Input fields taken from the `record` are:

- `Aid element £`

#### Parameters

- `record` **[Object][3]** The row received from parsed file

Returns **Budget**

### getDescription

Preprocess `description`.

Input fields taken from the `record` are:

- `Beneficiary identifier (E-claims ref)`
- `Sector NACE group level`
- `SANI reference of the aid measure`
- `Objective of the aid`

#### Parameters

- `record` **[Object][3]** The row received from parsed file

Returns **[String][4]**

### getProjectId

Preprocess `project_id`.

Input fields taken from the `record` are:

- `Objective of the aid`

#### Parameters

- `record` **[Object][3]** The row received from parsed file

Returns **[String][4]**

### getLocations

Preprocess `project_locations`.

Input fields taken from the `record` are:

- `Location of Benficiary NUTS level II (drop down)`

#### Parameters

- `record` **[Object][3]** The row received from parsed file

Returns **[Array][5]&lt;[Location][6]>**

### getThirdParties

Preprocess `third_parties`.

Input fields taken from the `record` are:

- `Name of beneficiary`

#### Parameters

- `record` **[Object][3]** The row received from parsed file

Returns **[Array][5]&lt;ThirdParty>**

### getTimeframe

Preprocess `timeframe`.

Input fields taken from the `record` are:

- `Date of granting`

#### Parameters

- `record` **[Object][3]** The row received from parsed file

Returns **Timeframe**

[1]: https://github.com/ec-europa/eubfr-data-lake/blob/master/services/ingestion/etl/2014uk16rfop001/ods/test/stubs/ESF/record.json
[2]: https://github.com/ec-europa/eubfr-data-lake/blob/master/services/ingestion/etl/2014uk16rfop001/ods/src/lib/transform/ESF/transform.js
[3]: https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Object
[4]: https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/String
[5]: https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Array
[6]: https://developer.mozilla.org/docs/Web/API/Location
Loading