Broken withRetry()'s "stopAfterAttempt" parameter, keep defaulting to 3 even when I specify a different number

[knamnguyen]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

const chain = RunnableSequence.from([
    ChatPromptTemplate.fromTemplate(prompt),
    model,
    parser,
]).withRetry({
        onFailedAttempt: (error) => {
            console.log(
                `[LLMChainWithRetry] Attempt ${error.attemptNumber} failed. There are ${error.retriesLeft} retries left.`,
            );
            throw Error(
                `Failed to parse the output at attempt ${error.attemptNumber}, there are ${error.retriesLeft} retries left`,
            );
            return `Failed to parse the output at attempt ${error.attemptNumber}, there are ${error.retriesLeft} retries left`;
        },
        stopAfterAttempt: 2, //I have tried different number here, including 2,5, and 10
    })
    .withConfig({ maxConcurrency: 20 });

Description

I'm trying to set up a Runnable with an output parser. I have tried setting different numbers for the "stopAfterAttempt" field for withRetry(), but when I invoke the chain with

 const result = await chain.batch(
        batchInput,
        {
            timeout: 1000000,
        },
        {
            returnExceptions: true,
        },
    );

the following exception is returned:

[
  {
    "llmOutput": "```json\n{\n  \"error\": \"Invalid JSON format. The JSON object must be an array of question groups, but the structure provided does not conform to the required schema. Each question should be encapsulated in its own object with the necessary properties. Additionally, the content must be properly escaped, and the overall structure must adhere to the specified format. To fix this, ensure that each question is represented as an object within an array, with all required fields filled out correctly, and that all special characters in the MMD content are escaped properly.\"\n}\n```",
    "sendToLLM": false,
    "originalLine": 40,
    "originalColumn": 10,
    "input": {...},
    "attemptNumber": 3,
    "retriesLeft": 0
  }
]

When I check the trace on langsmith it always retries exactly 3 times, no matter what value I set for stopAfterAttempt.

Appreciate anyone who can help!

System Info

platform windows
node 20.12.2
pnpm 9.10.0

[email protected] | MIT | deps: 12 | versions: 297
Typescript bindings for langchain
https://github.com/langchain-ai/langchainjs/tree/main/langchain/

keywords: llm, ai, gpt3, chain, prompt, prompt engineering, chatgpt, machine learning, ml, openai, embeddings, vectorstores

dist
.tarball: https://registry.npmjs.org/langchain/-/langchain-0.3.2.tgz
.shasum: aec3e679d3d6c36f469448380affa475c92fbd86
.integrity: sha512-kd2kz1cS/PIVrLEDFlrZsAasQfPLbY1UqCZbRKa3/QcpB33/n6xPDvXSMfBuKhvNj0bjW6MXDR9HZTduXjJBgg==
.unpackedSize: 2.9 MB

dependencies:
@langchain/openai: >=0.1.0 <0.4.0 js-yaml: ^4.1.0 openapi-types: ^12.1.3 yaml: ^2.2.1
@langchain/textsplitters: >=0.0.0 <0.2.0 jsonpointer: ^5.0.1 p-retry: 4 zod-to-json-schema: ^3.22.3
js-tiktoken: ^1.0.12 langsmith: ^0.1.56-rc.1 uuid: ^10.0.0 zod: ^3.22.4

maintainers:

• nfcampos [email protected]
• jacoblee93 [email protected]
• andrewnguonly [email protected]
• davidduong [email protected]
• hwchase17 [email protected]
• basproul [email protected]

dist-tags:
latest: 0.3.2 next: 0.3.2-rc.0 tag-for-publishing-older-releases: 0.2.20

published 3 weeks ago by jacoblee93 [email protected]

[dosubot[bot]]

Hey there, @knamnguyen! 👋 I'm here to help you with your code snags and answer any questions you have. I can also guide you on how to contribute. Let's squash those bugs together!

The issue you're experiencing, where the stopAfterAttempt parameter in the withRetry function defaults to 3 despite specifying a different number, could be due to several reasons. Here are a few things you might want to check:

1. Parameter Passing: Ensure that the stopAfterAttempt parameter is being correctly passed to the withRetry function. Double-check that there are no typos or issues in the way the parameter is set in your code.

2. Code Logic: Verify that there is no other part of your code that might be resetting or overriding the stopAfterAttempt value after it is set. This could be due to a bug or unintended logic elsewhere in your code.

3. Library Version: Make sure you are using the latest version of the LangChainJS library. There might have been updates or bug fixes related to this functionality.

4. Internal Overrides: Although there is no documented configuration or global setting in LangChainJS that might override the stopAfterAttempt parameter, it's worth checking if there are any recent changes or commits in the codebase that might have affected the behavior of the withRetry function.

If none of these steps resolve the issue, consider creating a minimal reproducible example and testing it in isolation to see if the problem persists. This can help identify if the issue is specific to your environment or setup.

To continue talking to Dosu, mention @dosu.

[knamnguyen]

@dosu I have checked that the stopAfterAttempt is correct per langchainjs syntax for withRetry(), there seems to be no typos. No other part in my code is resetting or overidding the value. I'm using the latest LangchainJS library as well. What might be wrong here?

[dosubot[bot]]

The issue you're experiencing with the stopAfterAttempt parameter defaulting to 3, despite specifying a different number, might be related to the internal logic of the RunnableRetry class. The withRetry function is supposed to set the maxAttemptNumber based on the stopAfterAttempt parameter you provide ^[1].

Here are a few additional steps you can take to troubleshoot:

1. Inspect the RunnableRetry Class: Check if there are any conditions or logic within the RunnableRetry class that might override the stopAfterAttempt parameter. Although I couldn't find specific details on this, it's worth reviewing the class implementation to ensure there are no hardcoded values or defaults being applied.

2. Test with a Minimal Example: Create a minimal example that isolates the use of withRetry to see if the issue persists. This can help determine if the problem is specific to your current setup or environment.

3. Review Recent Changes: If possible, review recent changes or commits in the LangChainJS repository that might have affected the withRetry function's behavior. There might be undocumented changes impacting its functionality.

If these steps don't resolve the issue, consider reaching out to the maintainers or checking for any open issues in the repository that might be related to this behavior.

To continue talking to Dosu, mention @dosu.

[knamnguyen]

@dosu the code works and retry the correct number of timeswhen I reduced the code to

export const chain= RunnableSequence.from([
    ChatPromptTemplate.fromTemplate(prompt),
    model,
    parser,
]).withRetry({
    stopAfterAttempt: 5,
});

Looks like the issue in my original code is that I tried to add a withConfig() after a withRetry(). Is this not the right way to add default config to my Runnable?

[dosubot[bot]]

Chaining withConfig() after withRetry() should generally work, as each method is designed to return a new Runnable instance with the updated configuration or retry logic. However, if you're experiencing issues, it might be due to how these methods interact in your specific setup.

The withConfig() method merges the provided configuration with the existing one, while withRetry() adds retry capabilities to the Runnable ^[1]. If adding withConfig() after withRetry() causes unexpected behavior, ensure that the configuration settings in withConfig() do not inadvertently affect the retry logic.

To troubleshoot, you can:

1. Check Configurations: Ensure that the configurations you are passing to withConfig() do not conflict with or override any settings related to retries.

2. Order of Methods: Although the order should not matter, try reversing the order of withConfig() and withRetry() to see if it affects the behavior.

3. Minimal Example: Test with a minimal example that includes both withRetry() and withConfig() to isolate the issue.

If the problem persists, it might be worth reviewing the specific configurations you're applying to see if they might be causing the retry logic to reset or behave unexpectedly.

To continue talking to Dosu, mention @dosu.

[knamnguyen]

@dosu Is it possible that my auto-formatting feature on the ide is messing up the syntax causing the code not to work?

It looks like withRetry() should be directly after the Runnable and not on a new line

[dosubot[bot]]

Could you provide more details about the auto-formatting settings in your IDE? Specifically, it would be helpful to know which IDE you are using and any specific formatting rules or plugins that might be affecting your code. Additionally, if you could share the exact code snippet as it appears in your IDE, that would help in identifying any potential syntax issues caused by formatting.

To continue talking to Dosu, mention @dosu.

[knamnguyen]

@dosu I'm using cursor. The formated code looks like the below. I just copied and pasting without any modifications:

export const parsedMmdChain = RunnableSequence.from([
    ChatPromptTemplate.fromTemplate<InputParseMmd>(prompt),
    model,
    parser,
])
    .withRetry({
        stopAfterAttempt: 2,
    })
    .withConfig({
        maxConcurrency: 20,
    });

[dosubot[bot]]

The formatted code you provided looks correct, and the chaining of withRetry() and withConfig() should work as expected. The issue is unlikely to be caused by the auto-formatting in your IDE, as the syntax appears to be valid.

If you're still encountering issues, consider checking if there are any specific configurations or settings in your IDE that might affect how the code is interpreted or executed. Additionally, ensure that there are no other parts of your code or environment that might be interfering with the behavior of the RunnableSequence.

To continue talking to Dosu, mention @dosu.