In this experiment we are eliciting domain knowledge from people to help in designing good predictors of product ratings. The task for the machine learning system is to predict product rating based on a textual review given by an Amazon user. Your task in this experiment is to help the machine to judge the degree of relevance of each keyword in predicting the product rating.
We have listed 70 of the most frequent keywords in the reviews of kitchen products. You should judge the probability of their relevance for predicting ratings based on textual reviews. You can do this by adjusting the probability value between 0 (not-relevant at all) to 1 (absolutely relevant) by moving a slider. Your feedback is only recorded when the slider is moved. Please try your best to give feedback to all keywords and only skip those that you are very uncertain about.
As an example, the keywords "must buy" or "disaster" provide useful information about the product rating, and therefore, they can be considered as relevant. On the other hand, the keyword "is" in the review text may not be very informative for rating prediction.
The expected time for completing the form is around 10 minutes.
In this experiment we are eliciting domain knowledge from people to help in designing good predictors of product ratings. The task for the machine learning system is to predict product rating based on a textual review given by an Amazon user. Your task in this experiment is to help the machine to judge the degree of relevance of each keyword in predicting the product rating.
We have listed 70 of the most frequent keywords in the reviews of kitchen products. You should judge the probability of their relevance for predicting ratings based on textual reviews. You can do this by adjusting the probability value between 0 (not-relevant at all) to 1 (absolutely relevant) by moving a slider. Your feedback is only recorded when the slider is moved. Please try your best to give feedback to all keywords and only skip those that you are very uncertain about.
As an example, the keywords "must buy" or "disaster" provide useful information about the product rating, and therefore, they can be considered as relevant. On the other hand, the keyword "is" in the review text may not be very informative for rating prediction.
The initial position of the slider, for each keyword, is selected by the machine. This is based on a machine learning algorithm that has access to 500 reviews and their corresponding ratings. In other words, this is the best that the machine could learn from 500 example reviews.
The expected time for completing the form is around 10 minutes.
