As a Sustainability apps user I would like to have a first MVP of the web-component implemented

[rcavaliere]

• integration of Suedtirol Radelt data
• review of reference design (for just this MVP - see detailed proposals below)
• application of MVP design
• integration in the Open Data Hub Web Components Store (with configurator of the design aspects)

Requirement: the filter "Sustainability Action" and "Organisation" should be considered even in the web-component and in the configuration of the web-component store.

Expected behavior:

• if in the web component configuration just a sustainability action is chosen -> the web-component does not present the sustainability action filter
• if in the web component configuration just an organisation is chosen -> the web-component does not present the organisation filter
• if in the web component configuration a sustainability action and an organisation is chosen -> the web-component does not present any filter
• if in the web component configuration no filters are applied -> the web-component present both filters

Review of reference design:
240523_ConceptAPPReview_NOI.pdf

[rcavaliere]

Just a clarification: in the field "Organisationen" we expect to see the nr of organizations, by enlarging the details we just see here the nr. of organizations divided per type.

[rcavaliere]

Another comment: for the field "Teilnehmerranking" should not contain e.g. "Autonome Provinz Bozen". I would suggest here:

• no "Details ausblenden"
• possibility to change ref. criteria for the ranking (e.g. KM, CO2)
• search button
• ranking

Please also include the possibility to scroll down the ranking.

Also the search must be defined. What happens if no result is obtained? What happens if a research returns multiple results? What happens if a user then selects an organization? In this last scenario: ranking centered to the position of this organization, which is properly highlighted (e.g. other background color) in the ranking

[rcavaliere]

@Peter080202 I add also here the reference to the API calls:

• CompanyGamificationAction: challgenge data, specific per organization https://mobility.api.opendatahub.testingmachine.eu/v2/flat,node/CompanyGamificationAction/*/latest?where=sactive.eq.true&limit=-1
• GamificationAction: total challenge data - https://mobility.api.opendatahub.testingmachine.eu/v2/flat,node/GamificationAction/*/latest?where=sactive.eq.true&limit=-1

These end-points retrieve only the last value. For getting the history, there is another API call (check https://swagger.opendatahub.com/?url=https://mobility.api.opendatahub.com/v2/apispec)

[rcavaliere]

Short summary of what we have agreed today with @Peter080202, relevant also for @danielecanu:

First implementation iteration (next week - to be discussed during the meeting scheduled on 6.9):

• first integration of Open Data Hub API
• visualization of "basic" KPI (e.g. number of km traveled, CO2 emissions reduced, nr. of participants, nr. of organizations, ranking of the organizations)
• implementation of all filters stuff -> first adaptation of original design proposal

Second implementation iteration:

• additional details / plots. Based on the available data, we could have as detail for the KPIs "number of km traveled", "CO2 emissions reduced" the temporal evolution (not a yearly basis based on the data available, but e.g. on a monthly basis)
• additional integration of Open Data Hub -> another API call must be used in order to retrieve the time series of the KPI evolution during time

[rcavaliere]

@Peter080202 @danielecanu as agreed today, following points should be fixed in order to complete a first MVP with Suedtirol Radelt data:

• add the "Challenge" filter (in the PDF called "Sustainability Action" filter, e.g. "Alto Adige Pedala 2023")
• hide the filters related to the different APPs (Suedtirol Radelt, ummadum, Loc.All)
• allow the ranking to be sorted according to different KPIs. Relevant KPI made available via API: km (API parameter: "km_total"), CO2 (API parameter: "co2"), participants (API parameter: "number_of_people"), money saved (API parameter: "money_saved")
• on the left we have four different boxes, two on the top and two on the bottom. Each box refers to one of the four KPIs.
• when an organization is clicked in the ranking, the ranking can not be scrolled up and down, the organization remains highlighted and the proper KPIs are updated in the four boxes in the left. By default, the boxes show the total values for all companies.
• if the filter "Challenge" is not applied, it is possible to see in the boxes the data for each challenge. Please consider in this first implementation iteration just the list view, the plot with time series will be implemented at a later stage. Please note that if a Challenge is selected in the filter, then we will have just the four numbers in the box, without any details below each box.

We also decided to leave the organization filter out, since it is already possible to search in the ranking and select one certain organization.

For the PR, it is probably better that you pull the application that we can then directly integrate and configure in the web-components store. Please include all relevant variables that can be set in the configuration of the web-component, including the "Challenge"

[Peter080202]

@rcavaliere
I have finished implementing the above points. I have two questions:
How to retrieve the challenges for the "Sustainability Action" filter e.g. "Alto Adige Pedala 2023"? Is there an existing endpoint that returns all available challenges?
How to apply the action filter to the API calls? In specific I mean these calls:
https://mobility.api.opendatahub.testingmachine.eu/v2/flat,node/CompanyGamificationAction/*/latest?where=sactive.eq.true&limit=-1
and
https://mobility.api.opendatahub.testingmachine.eu/v2/flat,node/GamificationAction/*/latest?where=sactive.eq.true&limit=-1

[rcavaliere]

@Peter080202 at present it seems that we are importing only the data from the challenge 2024, but we are still checking the data flow. However the rule would be the following:

• in CompanyGamificationAction, the challenge name is appended to sname, e.g. "Gruppo Sportivo Fortezza-Südtirol radelt 2024" (i.e. CompanyName-ChallengeName)
• in GamificationAction, it is always in sname (in this case is bilingual, i.e. nameDE_nameIT)

In generale, the latter method returns all available Challenge, you should distinguish the different "stations" provided here. You could use also a different API call, if you want to have just this list.

[rcavaliere]

@Peter080202 in the meantime, we have solved the issue with the data flow, now it seems that everything runs in a stable way.
In order to get the timeseries of a certain KPI related to a certain company, consider e.g. this API call:

https://mobility.api.opendatahub.testingmachine.eu/v2/flat,node/CompanyGamificationAction/km_average/2024-09-21T00:00:00.000Z/2024-09-24T23:59:59.999Z?limit=-1&where=sname.eq.Sasa%20SpA-S%C3%BCdtirol%20radelt%202024

With this call we have asked the data of the last three days (21.09 - 24.09) related to the KPI km_average related to the company Sasa SpA-Südtirol radelt 2024.

[Peter080202]

@rcavaliere , I just submitted the pull request for the first version. The PR doesn't include any logic for generating charts as I encountered an issue with the provided API. Since we are visualising data for e.g. Südtirol Radelt 2024, I believe it would make sense to call the endpoint using a start date of 2024-01-01T00:00:00.000Z and an end date of 2024-12-31T23:59:59.999Z. However, the API currently only supports a date range of up to 5 days. Could this limit be extended, perhaps to 31 days, so that we can at least fetch data month by month and aggregate it, resulting in 12 requests per year?

Additionally, @danielecanu suggested that summing up the values for the entire year may not be the best approach, as we could be in July and Südtirol Radelt 2024 might only return data until June 2024, which will probably lead to a dent in the curve. He proposed that it may be more appropriate to draw separate lines for each action, resulting in x lines where x equals the number of sustainability actions tracked. You can discuss this idea further in the next meeting.

Lastly, during our previous meeting, I mentioned an API issue where some organisations appeared twice (Marktgemeinde St. Lorenzen and Libera Universitá di Bolzano - Freie Universität Bozen). I've attached a screenshot showing the data for Marktgemeinde St. Lorenzen:

Please reach out if you need any clarifications. Please also make sure to tag the developers who are taking over the project (@mathiasmader and @redek91).

Thanks!

[rcavaliere]

@Peter080202 thanks for the feedback, we will discuss this.
We also check in the data this issue with duplicated records. Have a nice future, hope to see you again soon or later!

[rcavaliere]

@mathiasmader @redek91 @danielecanu we have deployed the first version of the web-component here: https://webcomponents.opendatahub.testingmachine.eu/webcomponent/sustainability-apps

I think we have some issues with the graphical elements, e.g. we don't see the arrow left and rights so it is everything a little bit ugly at the moment. Don't we also have a basic graphical layout for the entire web-component to be used?

I kindly ask you to check this, you can refer to my colleague @clezag if we need to adapt something in our deployment. Apart from this, from a technical perspective everything is implemented as we agreed and the data is properly shown. One usability issue that I have currently seen is that if you try to select another organisation from the list in order to check the specific KPI associated to it, this is not possible. I would also like to have the possibility to not select any organisation at all and see the total KPIs.

One additional feature which would be nice to see is the historical evolution of the KPI. Now that we have a stable data flow, we have seen that in the back-end the KPIs are updated once a day, i.e. we have an updated measurement in correspondence of each day. This means that we could simply draw some plots in the empty part of the web-component by just showing for each calendar day the correspondent value. The API call to be used to retrieve this is to be found here: #1 (comment)

So by summarizing, before considering this first implementation iteration closed, I would request you the following:

• Check current issues with the graphical layout of the web-component, which seems to be not correctly deployed
• Solve the usability issue with the selection of the organization
• Implement the visualization of historical values

Can you provide us an estimation for checking and implementing all this? We could also organize a short meeting if needed, or to better explain all these point or (if everything is clear) once a new Pull Request (PR) is made and deployed, so that we can discuss and finally approve all this and organize the next steps.

[redek91]

@rcavaliere I already opened a PR for the graphical issues. #6

[rcavaliere]

@redek91 thanks for the notice, we integrated the PR and now can see the graphical elements correctly. What about the overall possibility of personalizing the layout of the web-component?

[redek91]

@rcavaliere I will make an estimation of the work for the display of the historical data.

I think there is a misunderstanding on how the component is working right now:

• The selection of the organization in the configuration only sets the organization that is shown on the top of the 3rd column, and does not filter the other KPIs My mistake, the KPIs are in fact filtered, but should be make the list items selectable and filter the data by the selected item?
• As for the layout, I can't see any setting that changes the view of the KPIs beside the font color, should there be a list of specific layouts?

We should organize a meeting with @danielecanu and @mathiasmader to clarify these points.

[redek91]

A solution to the usability issue could be to have a organization preselected and a dropdown with all the organizations and filter by the selected one, the selection could also be removed. And keep the list of all organization unfiltered and highlight the selected one. Something like this

[rcavaliere]

@redek91 yes, I like your idea, actually we already discussed it. From an usability point of view, I think it's the best choice possible.

[rcavaliere]

@redek91 can we plan to have the organization filter view implemented this week? And maybe the plots with the historical data next week? Let me know about a possible implementation plan, so that we can have a first complete PoC

[rcavaliere]

@redek91 @danielecanu as agreed, let me summarize the list of modifications we agreed today:

• complete the implementation of the "organization" filter (already developed, as presented in your dev environment)
• freeze the four "blocks" in the left part of the application, one for each type:

1. kilometers traveled
2. participants
3. money saved
4. CO2 saved

This means that it won't be possible anymore to change among types, as actually the case.

• Below each block if "Show details" is enabled it is possible to see the plot. We decided to have one value per week, and to show all data available
• the ranking on the right side should be changed as follows: "Overall ranking" above, type below, with the possibility to change among the available types. Let's add a label "Leader: " in the box indicating the top ranking organization (e.g. "Leader. Gemeinde Bozen - Comune di Bolzano
• in the ranking, let's add the ranking number of the organisations, so it's clearer how good an organisation is.

Plan is to complete all these missing developments within week 47. After this we will concentrate on the possibility to apply a certain graphical layout to the web component in the configurator

[redek91]

@rcavaliere I think we have a problem with the timeseries as it is right now if we want to show 1 value for each week.

Currently the api gives us values over time for each of (km_total,co2,money_saved, number_of_people) per day.
The api only allows to retrieve 5 days per request and this request takes 2 seconds.

So if we want to get all data of the last year the formula is:
52 [weeks per year] * 7[days per week] / 5 [limitation of days the api imposes]

We get 72.8 also we need 73 requests to get the whole year.

Each request takes 2 seconds, also it would be 73 * 2 / 60 = 2,5 min
This is by any means feasible, unless it is ok that the data is loaded in background and is available just after 2 minutes.

Would it be possible to provide an endpoint that already provides such data?
A possible solution could be an endpoint that takes following parameters and returns the aggregated dataset:

• sustainability action
• organization (optional, all if not given)
• value type [km_total, co2, etc...] (optional, all if not given)
• the standard filter and select fields
• granularity (week, day, month)

This way the data would be aggregated on DB side, which is probably the most efficient way.

Can we make a call to solve this? Please let me know.

[rcavaliere]

@redek91 I think this is somehow doable. We have for example this application (https://analytics.opendatahub.com/) with which we can see large timeseries of data without any particular issue. Check for example this API call:

https://analytics.opendatahub.com/#%7B%22active_tab%22:0,%22height%22:%22400px%22,%22auto_refresh%22:false,%22scale%22:%7B%22from%22:1719784800000,%22to%22:1729634400000%7D,%22graphs%22:%5B%7B%22category%22:%22Parking%22,%22station%22:%22108%22,%22station_name%22:%22P08%20-%20BZ%20Centro%20via%20Mayr%20Nusser%22,%22data_type%22:%22free%22,%22unit%22:%22%22,%22period%22:%22300%22,%22yaxis%22:1,%22color%22:1%7D%5D%7D

In your case you would have max 52 points per plot, so this shouldn't be an issue. Let's try to implement this, and then see how to eventually improve the performance

[redek91]

@rcavaliere this data is only for 1 parking station, as soon as you remove the scode filter it has the same problem. It is normal it is a log of data.

Is the day limitation only on the test env "https://mobility.api.opendatahub.testingmachine.eu"?

Should we show the graphs only if an organization is selected? In this case the data would only be loaded if the organization is selected or changed

[clezag]

Just chiming in, @rcavaliere this is related to our new history quota, which is only active in testing right now (production soon).

You can raise the limit to 100 days by passing a referer header, but beyond that you need an oauth token with appropriate service permissions.
On analytics, logging in to a user that has more permissions solves the issue, but that's not doable for webcomponents.

In general, it's not ideal that a webcomponent would request lots of historical data and do the aggregation on every request.
One solution could be a custom backend, as you proposed, which could then access using an oauth user, and also do some caching etc.
Another way could be that we implement an elaboration that aggregates the data and writes it back to the Open Data Hub, so there is less volume, but you'd still have to do n = [days in time window] / 100 requests

[rcavaliere]

@clezag thanks for the hint, I didn't thought at the quota topic. Let's discuss together what kind of solution would be better. @redek91 we will provide you a feedback ASAP, in the meantime please proceed on the other topics

[rcavaliere]

@redek91 I tested a little bit the new version of web-component, which you could also see (full screen) on https://webcomponents.opendatahub.testingmachine.eu/webcomponent/sustainability-apps

In general, very good work! We are approaching a final MVP. I just report here some few minor cosmetic things, which should be fixed in order to a presentable web component:

• the plots with the historical data look a little compressed on the vertical axis, the visualization is not optimal

• I would recommend to show data from the most recent day (above) to the historical ones (bottom), not the contrary as currently implemented

• the search box for the organization is too near to the first position ranking, there should be a little space there

• I would recommend to improve the visualization of the fact that there is a gap between the positions of the ranking (in the example below: first positions vs position of the company selected, near the bottom of the ranking - explain that we do not visualize all the positions in between

Can you please fix these points and then make us a new PR? I provide you then a feedback for the visualization of more historical values.

[rcavaliere]

@redek91 very good, I think we can consider complete this very first implementation!

[redek91]

@redek91 very good, I think we can consider complete this very first implementation!

Nice to hear that. Thank you for the feedback.