Are JSON Schema tools actually supported, or only Zod?

[dwabyick]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

return new DynamicStructuredTool({
        name: definition.agentActionId,
        description: definition.fullDescription!,
        schema: (definition.inputs?.properties as Record<string, any>) ?? {},
        func: async (
          args: Record<string, unknown>
        ): Promise<AssistantActionInvocation> => {
          // Just return the invocation for now.
          return {
            invocationId: uuidv4(),
            agentActionId: definition.agentActionId,
            inputs: args,
          };
        },
      });

Description

I'm trying to start using an existing OpenAI tool-calling API over to Langchain. We have our tools defined as JSON schema only.

  "inputs": {
      "type": "object",
      "properties": {
        "canvasSelection": {
          "type": "array",
          "description": "The IDs of the scene nodes to edit",
          "items": { "type": "string" }
        }
      },
      "required": ["canvasSelection"]
    }

It seems like the schema is completely thrown away by the following code in tools/index.ts

 this.schema = (
      isZodSchema(fields.schema) ? fields.schema : z.object({}).passthrough()

Perhaps there's some other way the schema gets passed to the LLM that I'm missing? The tutorials all seem to use Zod.

System Info

Mac / Node 18

[dosubot[bot]]

Hey @dwabyick! I'm here to help you with any bugs, questions, or contributions while you're waiting for a human maintainer. Let's get to work!

JSON Schema tool descriptions are indeed supported in LangChain. The documentation specifies that the schema for tools can be provided as either Zod or JSON schema. However, it's important to note that if you pass a JSON schema, tool inputs will not be validated ^[1].

In your example, it seems like the schema might be getting overridden by a default Zod schema. This could be due to the isZodSchema function, which checks if the provided schema is a Zod schema and defaults to a passthrough Zod object if it is not ^[2].

To ensure your JSON schema is used correctly, you might need to verify how the schema is being passed and utilized in your specific implementation. If the tutorials primarily use Zod, it might be beneficial to adapt your JSON schema to Zod for validation purposes, or ensure that your JSON schema is correctly integrated into the tool setup without being overridden.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[dwabyick]

This isn't super helpful @dosu, as I pretty much mentioned the overridden code in my example above.

Can you provide a working example that uses JSON schema? In particular, one that correctly integrated into the tool setup without being overridden? We don't need schema validation, just to ensure that the JSON schemas get provided to the LLM.

[dosubot[bot]]

To use JSON schemas in LangChain without them being overridden by default Zod schemas, you can directly pass the JSON schema to the tool setup. Here's an example that demonstrates how to integrate a JSON schema into a DynamicStructuredTool without requiring schema validation:

import { DynamicStructuredTool } from 'langchain-core/src/tools';
import { v4 as uuidv4 } from 'uuid';

// Define your JSON schema
const jsonSchema = {
  type: "object",
  properties: {
    canvasSelection: {
      type: "array",
      description: "The IDs of the scene nodes to edit",
      items: { type: "string" }
    }
  },
  required: ["canvasSelection"]
};

// Create a DynamicStructuredTool using the JSON schema
const tool = new DynamicStructuredTool({
  name: "exampleTool",
  description: "A tool that uses JSON schema",
  schema: jsonSchema, // Pass the JSON schema directly
  func: async (args: Record<string, unknown>) => {
    // Just return the invocation for now.
    return {
      invocationId: uuidv4(),
      agentActionId: "exampleAction",
      inputs: args,
    };
  },
});

// Use the tool as needed in your application

In this setup, the JSON schema is directly assigned to the schema property of the DynamicStructuredTool. This ensures that the schema is passed to the LLM without being overridden by a default Zod schema. Since you mentioned that schema validation is not required, this approach focuses on ensuring the JSON schema is correctly integrated and provided to the LLM.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[dwabyick]

This seems to be incorrect. Passing in schema to DynamicStructuredTool will get overridden as isZodSchema will return false in the constructor.

constructor(fields: DynamicStructuredToolInput<T>) {
    super(fields);
    this.name = fields.name;
    this.description = fields.description;
    this.func = fields.func;
    this.returnDirect = fields.returnDirect ?? this.returnDirect;
    this.schema = (
      isZodSchema(fields.schema) ? fields.schema : z.object({}).passthrough()
    ) as T extends ZodObjectAny ? T : ZodObjectAny;
  }

[dwabyick]

Ok, it seems like I was on the wrong track with DynamicStructuredTool and JSON schema.

The following doesn't really provide a Langchain "tool" with a function to invoke, but it does at least expose the functions to the LLM, which is what I was trying to achieve.

const tools = [{
     type: "function",
     function: {
       name: "myFunctionName",
       description: "myFunctionDescription"
       parameters: { 
            type: "object",
            properties: {
                propName: {...}
            }
       }
    }
}];
const modelWithTools = this.model.bindTools(tools);

const response = await modelWithTools.invoke([
   { role: "system", content: systemPrompt },
   { role: "user", content: userMessage },
]);