prompts.json

{
    "io1-topk-v2": {
        "pre_text": "Context: \"We are analyzing a fixed set of perturbations around a specific input to understand the influence of each feature on the model's output. The dataset below contains the features 'A' through '{final_feature}' and the corresponding outputs.\n\nDataset:\n```",
        "mid_text": "",
        "post_text": "```\n\nQuestion: \"Based on the model's predictions and the given dataset, what appears to be the top {k_word} most important features in determining the model's prediction?\"\n\nInstructions: \"Think about the question. After explaining your reasoning, provide your answer as the top {k_word} most important features ranked from most important to least important, in descending order. Only provide the feature names on the last line. Do not provide any further details on the last line.\""
    },
    "pe2-topk-v2": {
        "pre_text": "Context: \"We are analyzing a fixed set of perturbations around a specific input to understand the influence of each feature on the model's output. The dataset below contains the features 'A' through '{final_feature}' and the corresponding outputs.\n\nDataset:\n```",
        "mid_text": "",
        "post_text": "```\n\nQuestion: \"Based on the model's predictions and the given dataset, estimate the output for the final input.\"\n\nInstructions: \"Think about the question. After explaining your reasoning provide your answer as:\n\na) on the penultimate line, the estimated output\n\nb) on the last line, the top {k_word} most important features ranked from most important to least important, in descending order.\n\nOnly provide the output and the feature names on the last two lines. Do not provide any further details on the last two lines.\""
    },
    "pfp2-v2": {
        "pre_text": "Context: \"We are analyzing a fixed set of perturbations around a specific input to understand the influence of each feature on the model's output. The dataset below contains the features 'A' through '{final_feature}' and the corresponding outputs.\n\nDataset:\n```",
        "mid_text": "",
        "post_text": "```\n\nFor each feature, starting with 'A' and continuing to '{final_feature}':\n\n1. Analyze the feature in question:\na. Compare instances where its changes are positive to where its changes are negative and explain how this difference correlates with the change in output.\nb. Rate the importance of the feature in determining the output on a scale of 0-100, considering both positive and negative correlations. Ensure to give equal emphasis to both positive and negative correlations and avoid focusing only on absolute values.\n\n2. After analyzing the feature, position it in a running rank compared to the features already analyzed. For instance, after analyzing feature 'B', determine its relative importance compared to 'A' and position it accordingly in the rank (e.g., BA or AB). Continue this process until all features from 'A' to '{final_feature}' are ranked.\n\nUpon completion of all analyses, provide the final rank of features from 'A' to '{final_feature}' on the last line.\n\nAvoid providing general methodologies or suggesting tools. Justify your findings as you go."
    },
    "pfp-io1-topk": {
        "pre_text": "Context: \"We are analyzing a fixed set of perturbations around a specific input to understand the influence of each feature on the model's output. The dataset below contains the change in features 'A' through '{final_feature}' (with negative values denoting a decrease in a feature's value) and the corresponding change in outputs.\n\nDataset:\n```",
        "mid_text": "",
        "post_text": "```\n\nQuestion: \"Based on the model's predictions and the given dataset, what appears to be the top {k_word} most important features in determining the model's prediction?\"\n\nInstructions: \"Think about the question. After explaining your reasoning, provide your answer as the top {k_word} most important features ranked from most important to least important, in descending order. Only provide the feature names on the last line. Do not provide any further details on the last line.\""
    },
    "pfpe2-topk": {
        "pre_text": "Context: \"We are analyzing a fixed set of perturbations around a specific input to understand the influence of each feature on the model's output. The dataset below contains the change in features 'A' through '{final_feature}' (with negative values denoting a decrease in a feature's value) and the corresponding change in outputs.\n\nDataset:\n```",
        "mid_text": "",
        "post_text": "```\n\nQuestion: \"Based on the model's predictions and the given dataset, estimate the change in output for the final change in input.\"\n\nInstructions: \"Think about the question. After explaining your reasoning provide your answer as:\n\na) on the penultimate line, the estimated change in output\n\nb) on the last line, the top {k_word} most important features ranked from most important to least important, in descending order.\n\nOnly provide the change in output and the feature names on the last two lines. Do not provide any further details on the last two lines.\""
    },
    "pfp2": {
        "pre_text": "We are analyzing a fixed set of perturbations around a specific input to understand the influence of each feature on the model's output. The dataset below contains the change in features 'A' through '{final_feature}' (with negative values denoting a decrease in a feature's value) and the corresponding change in outputs.\n\nDataset:\n```",
        "mid_text": "",
        "post_text": "```\n\nFor each feature, starting with 'A' and continuing to '{final_feature}':\n\n1. Analyze the feature in question:\na. Compare instances where its changes are positive to where its changes are negative and explain how this difference correlates with the change in output.\nb. Rate the importance of the feature in determining the output on a scale of 0-100, considering both positive and negative correlations. Ensure to give equal emphasis to both positive and negative correlations and avoid focusing only on absolute values.\n\n2. After analyzing the feature, position it in a running rank compared to the features already analyzed. For instance, after analyzing feature 'B', determine its relative importance compared to 'A' and position it accordingly in the rank (e.g., BA or AB). Continue this process until all features from 'A' to '{final_feature}' are ranked.\n\nUpon completion of all analyses, provide the final rank of features from 'A' to '{final_feature}' on the last line.\n\nAvoid providing general methodologies or suggesting tools. Justify your findings as you go."
    },
    "icl_exp": {
        "pre_text": "",
        "mid_text": "\n",
        "post_text": "\n",
        "backup": "Using the above sets of input-output pairs, find the importance of all {k_word} features in predicting the output of the inputs. Use comma separated value format for the generation and, starting with the most important feature, state your answer in descending order. Don't provide any further descriptions besides your answer."
    },
    "explain": {
        "pre_text": "Context: \"We have a two-class machine learning model that predicts based on {num_features} features: {feat_words}. The dataset below contains the{change_in} feature values 'A' through '{final_feature}'{wrt} and the corresponding{change_in} model outputs.\"\n\nDataset:\n```",
        "mid_text": "",
        "post_text": "```\n\nQuestion: \"Based on the above set, what are the {k_word} most important features driving the output?\"\n\nInstructions: \"Think about the question. {reasoning}rovide your answer as the top {k_word} features ranked from most important to least important, in descending order, separated by commas. Only provide the feature names{last_line}. Do not provide any further details{last_line}.\""
    },
    "explain_no_context": {
        "pre_text": "Dataset:\n```",
        "mid_text": "",
        "post_text": "```\n\nQuestion: \"Based on the above set, what are the {k_word} most important features driving the output?\"\n\nInstructions: \"Think about the question. {reasoning}rovide your answer as the top {k_word} features ranked from most important to least important, in descending order, separated by commas. Only provide the feature name{last_line}. Do not provide any further details{last_line}.\""
    },
    "predict_then_explain": {
        "pre_text": "Context: \"We have a two-class machine learning model that predicts based on {num_features} features: {feat_words}. The dataset below contains the{change_in} feature values 'A' through '{final_feature}'{wrt} and the corresponding{change_in} model outputs.\"\n\nDataset:\n```",
        "mid_text": "",
        "post_text": "```\n\nQuestion: \"Based on the above set, estimate the{change_in} output for the final{change_in} input. What are the {k_word} most important features driving the output?\"\n\nInstructions: \"Provide the estimated{change_in} output immediately as a single integer on the first line of your response. Do not provide any further details on the first line. Think about the question. {reasoning}rovide your answer the top {k_word} features ranked from most important to least important, in descending order, separated by commas. Only provide the feature names{last_line}. Do not provide any further details{last_line}.\""
    },
    "explain_with_instructions": {
        "pre_text": "Context: \"We have a two-class machine learning model that predicts based on {num_features} features: {feat_words}. The dataset below contains the{change_in} feature values 'A' through '{final_feature}'{wrt} and the corresponding{change_in} model outputs.\"\n\nDataset:\n```",
        "mid_text": "",
        "post_text": "```\n\nQuestion: \"Based on the above set, what are the {k_word} most important features driving the output?\"\n\nInstructions: \"For each feature, starting with 'A' and continuing to '{final_feature}':\n\n1. Analyze the feature in question. Rate the importance of the feature in determining the output on a scale of 0-100, considering both positive and negative correlations. Ensure to give equal emphasis to both positive and negative correlations and avoid focusing only on absolute values.\n\n2. After analyzing the feature, position it in a running rank compared to the features already analyzed. For instance, after analyzing feature 'B', determine its relative importance compared to 'A' and position it accordingly in the rank (e.g., BA or AB). Continue this process until all features from 'A' to '{final_feature}' are ranked.\n\n{reasoning}rovide your answer as the final rank of features from 'A' to '{final_feature}' from most important to least important, in descending order, separated by commas. Only provide the feature names{last_line}. Do not provide any further details{last_line}.\""
    },
    "explain_with_instructions_no_context": {
        "pre_text": "Dataset:\n```",
        "mid_text": "",
        "post_text": "```\n\nQuestion: \"Based on the above set, what are the {k_word} most important features driving the output?\"\n\nInstructions: \"For each feature, starting with 'A' and continuing to '{final_feature}':\n\n1. Analyze the feature in question. Rate the importance of the feature in determining the output on a scale of 0-100, considering both positive and negative correlations. Ensure to give equal emphasis to both positive and negative correlations and avoid focusing only on absolute values.\n\n2. After analyzing the feature, position it in a running rank compared to the features already analyzed. For instance, after analyzing feature 'B', determine its relative importance compared to 'A' and position it accordingly in the rank (e.g., BA or AB). Continue this process until all features from 'A' to '{final_feature}' are ranked.\n\n{reasoning}rovide your answer as the final rank of features from 'A' to '{final_feature}' from most important to least important, in descending order, separated by commas. Only provide the feature names{last_line}. Do not provide any further details{last_line}.\""
    },
    "logprob": {
        "pre_text": "",
        "mid_text": "",
        "post_text": "Based on the above set, without any further explanation and only including the feature names, the {k_word} most important features driving the output, in descending order, are features"
    },
    "senti_classif_remove_words_v1": {
      "pre_text": "Context: \"We are analyzing a fixed set of word removals on a specific sentence to understand the influence on the model’s output. The dataset below contains the words removed from the original sentence and the corresponding change in output.\"\n\nDataset:\n```",
      "mid_text": "",
      "post_text": "```\n\nQuestion: \"Based on the model’s predictions and the given dataset, what appears to be the top three most important words in determining the model’s prediction?\"\n\nInstructions: \"Think about the question. After explaining your reasoning, provide your answer as the top three most important words ranked from most important to least important, in descending order. Only provide the important words on the last line. Do not provide any further details on the last line. Provide the answer on one line with each word separated by commas.\"\n"
    },
    "senti_classif_remove_words_pgicl": {
      "pre_text": "Context: \"We are analyzing a fixed set of word removals on a specific sentence to understand the influence on the model’s output. The dataset below contains the words removed from the original sentence and the corresponding change in output.\"\n\nDataset:\n```",
      "mid_text": "",
      "post_text": "```\n\nQuestion: \"Based on the above set, what are the top three most important words driving the output?\"\n\nInstructions: \"\n\n1. Analyze the word in question. Rate the importance of the word in determining the output on a scale of 0-100, considering both positive and negative correlations. Ensure to give equal emphasis to both positive and negative correlations and avoid focusing only on absolute values.\n\n2. After analyzing the word, position it in a running rank compared to the words already analyzed. For instance, after analyzing word 'B', determine its relative importance compared to word 'A' and position it accordingly in the rank (e.g., BA or AB). Continue this process until all words are ranked.\n\nAfter explaining your reasoning, provide your answer as the final rank of the words from most important to least important, in descending order, separated by commas. Only provide the words on the last line. Do not provide any further details on the last line.\""
    }
}