How do I apply OutputFixingParser to each failed attempt of a batch call? #6949
Replies: 1 comment
-
To apply
Here's an updated version of your code: const chain = RunnableSequence.from([
ChatPromptTemplate.fromTemplate(prompt),
model,
parser,
]).withRetry({
stopAfterAttempt: 15,
onFailedAttempt: async (error, input) => {
console.log(`Attempting fix. Failed attempt: ${error}`);
try {
// Attempt to fix the output using the fixingParser
const fixedOutput = await fixingParser.parse(input);
// If successful, return the fixed output to stop further retries
return fixedOutput;
} catch (fixError) {
// If fixing fails, log the error and allow retry to continue
console.log(`Fixing attempt failed: ${fixError}`);
throw error; // Re-throw the original error to trigger retry
}
},
});
const result = await chain.batch(
inputBatch,
{
timeout: 1000000, // 1000 seconds
maxConcurrency: 10,
},
{
returnExceptions: true,
},
); Key Points:
This setup ensures that the |
Beta Was this translation helpful? Give feedback.
-
Checked other resources
Commit to Help
Example Code
Description
I'm trying to batch many calls to gpt-4o-mini to parse text into a defined json using the StructuredOutputParser.
If one of the call inside the batch fails, it retries up to 15 times, which is great.
However, I'm trying to set up a flow where before it retries for a single input in a batch, it attempts to fix the output of that call first and parse it again, and only continues to retries if that fixingParser still returns a parsing error. This should happens each time before it decides to retry again - so it should try to call fixingParser 15 times for each call in the batch.
Right now in my above implementation, I tried to use onFailedAttempt to call the fixingParser, which is indeed called after each try and retry fails. However, it doesn't pause the retry flow to try to fix the output of the previous retry before continuing with the next (in case the fixing does not work). The fixing parser seems to be called independently from the batch and the retry keeps on going until the limit without every getting the output of the fixing parser.
How should I make it so that the fixingParser is applied after each failed attempt from a call in the batch?
System Info
platform Windows
node 20.12.2
pnpm 9.10.0
Beta Was this translation helpful? Give feedback.
All reactions