-
Notifications
You must be signed in to change notification settings - Fork 3.1k
feat: add XML tool calling support as provider setting #11973
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 1 commit
a397886
1ce143a
4269390
c0877f3
8a6d111
116061a
1977f54
fe27805
5e02378
f563508
5fbe17d
98d6d31
93b4f25
d021796
b999753
1407657
7ad6d0d
1969e48
2891cfe
0fe62ef
e63276b
2f5eb45
3f44db8
673cdbc
0a8ac63
2f2226e
ef1482a
5201e29
9fa6f0d
df96e99
7961bff
a7126a7
6b80262
a55ea3e
6fc24a7
effe896
a4af083
e136127
0a26055
1644707
6e3dd14
47aee65
83c9faa
4a2895b
6e80dcc
9c73980
4b4efcd
e1010ca
40f4e63
cd4ecc8
cc05554
30d5af2
1e43390
4bdf50c
9c9efa0
aba3af3
5a7d55f
8b5a571
175707c
bd3d185
5b82735
c04d89c
ce1faeb
2b90de6
94d9f9a
7c76d6f
b5f2859
1aa2eba
d7bf0c1
4784c50
ed966a1
d1c8798
7a73a84
c4a1958
1dd470f
0c3e9cf
1cf3e6f
cc24b3c
e666071
11941dd
b020f0d
bb21c14
a07d7ea
dda1a66
9b27f86
ac7481d
033d205
13e0d42
a282304
4b60c92
4f2203d
9edf375
ac1bd80
2158d18
161bcff
ba99070
33ffa3c
ca73a7c
17c4553
e7d910d
02b8581
135c26e
b8c6a13
8a20a6c
09dc855
0b05564
15a0fb6
d272928
94771f7
ec9027d
f037640
7d5a867
f78cb0b
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -75,10 +75,15 @@ export class AnthropicVertexHandler extends BaseProvider implements SingleComple | |||||||||||||||||||||||||||||||
| // Filter out non-Anthropic blocks (reasoning, thoughtSignature, etc.) before sending to the API | ||||||||||||||||||||||||||||||||
| const sanitizedMessages = filterNonAnthropicBlocks(messages) | ||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| const nativeToolParams = { | ||||||||||||||||||||||||||||||||
| tools: convertOpenAIToolsToAnthropic(metadata?.tools ?? []), | ||||||||||||||||||||||||||||||||
| tool_choice: convertOpenAIToolChoiceToAnthropic(metadata?.tool_choice, metadata?.parallelToolCalls), | ||||||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||||||
| // When useXmlToolCalling is enabled, omit native tool definitions from the API request. | ||||||||||||||||||||||||||||||||
| // The model will rely on XML tool documentation in the system prompt instead, | ||||||||||||||||||||||||||||||||
| // and output tool calls as raw XML text parsed by TagMatcher. | ||||||||||||||||||||||||||||||||
| const nativeToolParams = metadata?.useXmlToolCalling | ||||||||||||||||||||||||||||||||
| ? {} | ||||||||||||||||||||||||||||||||
| : { | ||||||||||||||||||||||||||||||||
| tools: convertOpenAIToolsToAnthropic(metadata?.tools ?? []), | ||||||||||||||||||||||||||||||||
| tool_choice: convertOpenAIToolChoiceToAnthropic(metadata?.tool_choice, metadata?.parallelToolCalls), | ||||||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||||||
|
Comment on lines
+78
to
+86
|
||||||||||||||||||||||||||||||||
| // When useXmlToolCalling is enabled, omit native tool definitions from the API request. | |
| // The model will rely on XML tool documentation in the system prompt instead, | |
| // and output tool calls as raw XML text parsed by TagMatcher. | |
| const nativeToolParams = metadata?.useXmlToolCalling | |
| ? {} | |
| : { | |
| tools: convertOpenAIToolsToAnthropic(metadata?.tools ?? []), | |
| tool_choice: convertOpenAIToolChoiceToAnthropic(metadata?.tool_choice, metadata?.parallelToolCalls), | |
| } | |
| // Always send native tool definitions to the API request so that tool calling | |
| // continues to work even when XML-based tool documentation is used elsewhere. | |
| const nativeToolParams = { | |
| tools: convertOpenAIToolsToAnthropic(metadata?.tools ?? []), | |
| tool_choice: convertOpenAIToolChoiceToAnthropic(metadata?.tool_choice, metadata?.parallelToolCalls), | |
| } |
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -75,10 +75,15 @@ export class AnthropicHandler extends BaseProvider implements SingleCompletionHa | |||||||||||||||||||||||||||||||||
| betas.push("context-1m-2025-08-07") | ||||||||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||
| const nativeToolParams = { | ||||||||||||||||||||||||||||||||||
| tools: convertOpenAIToolsToAnthropic(metadata?.tools ?? []), | ||||||||||||||||||||||||||||||||||
| tool_choice: convertOpenAIToolChoiceToAnthropic(metadata?.tool_choice, metadata?.parallelToolCalls), | ||||||||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||||||||
| // When useXmlToolCalling is enabled, omit native tool definitions from the API request. | ||||||||||||||||||||||||||||||||||
| // The model will rely on XML tool documentation in the system prompt instead, | ||||||||||||||||||||||||||||||||||
| // and output tool calls as raw XML text parsed by TagMatcher. | ||||||||||||||||||||||||||||||||||
| const nativeToolParams = metadata?.useXmlToolCalling | ||||||||||||||||||||||||||||||||||
| ? {} | ||||||||||||||||||||||||||||||||||
| : { | ||||||||||||||||||||||||||||||||||
| tools: convertOpenAIToolsToAnthropic(metadata?.tools ?? []), | ||||||||||||||||||||||||||||||||||
| tool_choice: convertOpenAIToolChoiceToAnthropic(metadata?.tool_choice, metadata?.parallelToolCalls), | ||||||||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||||||||
|
Comment on lines
+78
to
+86
|
||||||||||||||||||||||||||||||||||
| // When useXmlToolCalling is enabled, omit native tool definitions from the API request. | |
| // The model will rely on XML tool documentation in the system prompt instead, | |
| // and output tool calls as raw XML text parsed by TagMatcher. | |
| const nativeToolParams = metadata?.useXmlToolCalling | |
| ? {} | |
| : { | |
| tools: convertOpenAIToolsToAnthropic(metadata?.tools ?? []), | |
| tool_choice: convertOpenAIToolChoiceToAnthropic(metadata?.tool_choice, metadata?.parallelToolCalls), | |
| } | |
| // Always send native tool definitions for Anthropic so that tool_use blocks are produced. | |
| // The useXmlToolCalling flag is currently ignored here because the rest of the codebase | |
| // expects native tool_use events and does not support XML-based tool calling. | |
| const nativeToolParams = { | |
| tools: convertOpenAIToolsToAnthropic(metadata?.tools ?? []), | |
| tool_choice: convertOpenAIToolChoiceToAnthropic(metadata?.tool_choice, metadata?.parallelToolCalls), | |
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,31 +1,174 @@ | ||
| import { getSharedToolUseSection } from "../tool-use" | ||
| import { getToolUseGuidelinesSection } from "../tool-use-guidelines" | ||
|
|
||
| describe("getSharedToolUseSection", () => { | ||
| it("should include native tool-calling instructions", () => { | ||
| const section = getSharedToolUseSection() | ||
| describe("default (native) mode", () => { | ||
| it("should include native tool-calling instructions", () => { | ||
| const section = getSharedToolUseSection() | ||
|
|
||
| expect(section).toContain("provider-native tool-calling mechanism") | ||
| expect(section).toContain("Do not include XML markup or examples") | ||
| expect(section).toContain("provider-native tool-calling mechanism") | ||
| expect(section).toContain("Do not include XML markup or examples") | ||
| }) | ||
|
|
||
| it("should include multiple tools per message guidance", () => { | ||
| const section = getSharedToolUseSection() | ||
|
|
||
| expect(section).toContain("You must call at least one tool per assistant response") | ||
| expect(section).toContain("Prefer calling as many tools as are reasonably needed") | ||
| }) | ||
|
|
||
| it("should NOT include single tool per message restriction", () => { | ||
| const section = getSharedToolUseSection() | ||
|
|
||
| expect(section).not.toContain("You must use exactly one tool call per assistant response") | ||
| expect(section).not.toContain("Do not call zero tools or more than one tool") | ||
| }) | ||
|
|
||
| it("should NOT include XML formatting instructions", () => { | ||
| const section = getSharedToolUseSection() | ||
|
|
||
| expect(section).not.toContain("<actual_tool_name>") | ||
| expect(section).not.toContain("</actual_tool_name>") | ||
| }) | ||
|
|
||
| it("should return native instructions when useXmlToolCalling is false", () => { | ||
| const section = getSharedToolUseSection(false) | ||
|
|
||
| expect(section).toContain("provider-native tool-calling mechanism") | ||
| expect(section).not.toContain("<actual_tool_name>") | ||
| }) | ||
| }) | ||
|
|
||
| it("should include multiple tools per message guidance", () => { | ||
| const section = getSharedToolUseSection() | ||
| describe("XML tool calling mode", () => { | ||
| it("should include XML formatting instructions when useXmlToolCalling is true", () => { | ||
| const section = getSharedToolUseSection(true) | ||
|
|
||
| expect(section).toContain("<actual_tool_name>") | ||
| expect(section).toContain("</actual_tool_name>") | ||
| expect(section).toContain("Tool uses are formatted using XML-style tags") | ||
| }) | ||
|
|
||
| it("should NOT include provider-native tool-calling text when useXmlToolCalling is true", () => { | ||
| const section = getSharedToolUseSection(true) | ||
|
|
||
| expect(section).not.toContain("provider-native tool-calling mechanism") | ||
| expect(section).not.toContain("Do not include XML markup or examples") | ||
| }) | ||
|
|
||
| it("should include parameter tag syntax example when useXmlToolCalling is true", () => { | ||
| const section = getSharedToolUseSection(true) | ||
|
|
||
| expect(section).toContain("<parameter1_name>value1</parameter1_name>") | ||
| expect(section).toContain("<parameter2_name>value2</parameter2_name>") | ||
| }) | ||
|
|
||
| it("should include TOOL USE header when useXmlToolCalling is true", () => { | ||
| const section = getSharedToolUseSection(true) | ||
|
|
||
| expect(section).toContain("TOOL USE") | ||
| expect(section).toContain("You have access to a set of tools") | ||
| }) | ||
|
|
||
| it("should include new_task XML example", () => { | ||
| const section = getSharedToolUseSection(true) | ||
|
|
||
| expect(section).toContain("<new_task>") | ||
| expect(section).toContain("<mode>code</mode>") | ||
| expect(section).toContain("</new_task>") | ||
| }) | ||
|
|
||
| it("should include execute_command XML example", () => { | ||
| const section = getSharedToolUseSection(true) | ||
|
|
||
| expect(section).toContain("You must call at least one tool per assistant response") | ||
| expect(section).toContain("Prefer calling as many tools as are reasonably needed") | ||
| expect(section).toContain("<execute_command>") | ||
| expect(section).toContain("<command>npm run dev</command>") | ||
| expect(section).toContain("</execute_command>") | ||
| }) | ||
|
|
||
| it("should include IMPORTANT XML FORMATTING RULES section", () => { | ||
| const section = getSharedToolUseSection(true) | ||
|
|
||
| expect(section).toContain("IMPORTANT XML FORMATTING RULES") | ||
| expect(section).toContain("Every opening tag MUST have a matching closing tag") | ||
| expect(section).toContain("Do NOT use self-closing tags") | ||
| expect(section).toContain("Do NOT include JSON objects") | ||
| expect(section).toContain("Do NOT wrap tool calls in markdown code blocks") | ||
| }) | ||
|
|
||
| it("should include COMMON MISTAKES TO AVOID section", () => { | ||
| const section = getSharedToolUseSection(true) | ||
|
|
||
| expect(section).toContain("COMMON MISTAKES TO AVOID") | ||
| expect(section).toContain("Using JSON format") | ||
| expect(section).toContain("Missing closing tags") | ||
| expect(section).toContain("Using self-closing") | ||
| expect(section).toContain("Correct XML format") | ||
| }) | ||
|
|
||
| it("should include read_file correct example in common mistakes", () => { | ||
| const section = getSharedToolUseSection(true) | ||
|
|
||
| expect(section).toContain("<read_file>") | ||
| expect(section).toContain("<path>src/app.ts</path>") | ||
| expect(section).toContain("</read_file>") | ||
| }) | ||
| }) | ||
| }) | ||
|
|
||
| describe("getToolUseGuidelinesSection", () => { | ||
| describe("default (non-XML) mode", () => { | ||
| it("should include base guidelines without XML reinforcement", () => { | ||
| const section = getToolUseGuidelinesSection() | ||
|
|
||
| expect(section).toContain("# Tool Use Guidelines") | ||
| expect(section).toContain("Assess what information you already have") | ||
| expect(section).toContain("Choose the most appropriate tool") | ||
| expect(section).toContain("If multiple actions are needed") | ||
| }) | ||
|
|
||
| it("should NOT include single tool per message restriction", () => { | ||
| const section = getSharedToolUseSection() | ||
| it("should NOT include XML reinforcement when called without arguments", () => { | ||
| const section = getToolUseGuidelinesSection() | ||
|
|
||
| expect(section).not.toContain("You must use exactly one tool call per assistant response") | ||
| expect(section).not.toContain("Do not call zero tools or more than one tool") | ||
| expect(section).not.toContain("REMINDER: You MUST format all tool calls as XML") | ||
| expect(section).not.toContain("Formulate your tool use using the XML format") | ||
| }) | ||
|
|
||
| it("should NOT include XML reinforcement when useXmlToolCalling is false", () => { | ||
| const section = getToolUseGuidelinesSection(false) | ||
|
|
||
| expect(section).not.toContain("REMINDER: You MUST format all tool calls as XML") | ||
| expect(section).not.toContain("Formulate your tool use using the XML format") | ||
| }) | ||
| }) | ||
|
|
||
| it("should NOT include XML formatting instructions", () => { | ||
| const section = getSharedToolUseSection() | ||
| describe("XML tool calling mode", () => { | ||
| it("should include XML reinforcement guidelines when useXmlToolCalling is true", () => { | ||
| const section = getToolUseGuidelinesSection(true) | ||
|
|
||
| expect(section).toContain("Formulate your tool use using the XML format") | ||
| expect(section).toContain("REMINDER: You MUST format all tool calls as XML") | ||
| }) | ||
|
|
||
| it("should include XML-specific numbered steps", () => { | ||
| const section = getToolUseGuidelinesSection(true) | ||
|
|
||
| expect(section).toContain("4. Formulate your tool use using the XML format") | ||
| expect(section).toContain("5. After each tool use, the user will respond") | ||
| expect(section).toContain("6. ALWAYS wait for user confirmation") | ||
| }) | ||
|
|
||
| it("should still include base guidelines alongside XML reinforcement", () => { | ||
| const section = getToolUseGuidelinesSection(true) | ||
|
|
||
| expect(section).toContain("# Tool Use Guidelines") | ||
| expect(section).toContain("Assess what information you already have") | ||
| expect(section).toContain("Choose the most appropriate tool") | ||
| }) | ||
|
|
||
| it("should include explicit XML structure reminder", () => { | ||
| const section = getToolUseGuidelinesSection(true) | ||
|
|
||
| expect(section).not.toContain("<actual_tool_name>") | ||
| expect(section).not.toContain("</actual_tool_name>") | ||
| expect(section).toContain("<tool_name><param>value</param></tool_name>") | ||
| }) | ||
| }) | ||
| }) |
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -1,9 +1,18 @@ | ||||||||||||||||||||||||||||||||||||||
| export function getToolUseGuidelinesSection(): string { | ||||||||||||||||||||||||||||||||||||||
| export function getToolUseGuidelinesSection(useXmlToolCalling?: boolean): string { | ||||||||||||||||||||||||||||||||||||||
| const xmlReinforcement = useXmlToolCalling | ||||||||||||||||||||||||||||||||||||||
| ? ` | ||||||||||||||||||||||||||||||||||||||
| 4. Formulate your tool use using the XML format specified for each tool. The tool name becomes the outermost XML tag, with each parameter as a nested child tag. | ||||||||||||||||||||||||||||||||||||||
| 5. After each tool use, the user will respond with the result of that tool use. This result will provide you with the necessary information to continue your task or make further decisions. | ||||||||||||||||||||||||||||||||||||||
| 6. ALWAYS wait for user confirmation after each tool use before proceeding. Never assume the success of a tool use without explicit confirmation of the result from the user. | ||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||
| **REMINDER: You MUST format all tool calls as XML.** Do not use JSON, function-call syntax, or any other format. Each tool call must use the exact XML structure: \`<tool_name><param>value</param></tool_name>\`.` | ||||||||||||||||||||||||||||||||||||||
| : "" | ||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||
| return `# Tool Use Guidelines | ||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||
| 1. Assess what information you already have and what information you need to proceed with the task. | ||||||||||||||||||||||||||||||||||||||
| 2. Choose the most appropriate tool based on the task and the tool descriptions provided. Assess if you need additional information to proceed, and which of the available tools would be most effective for gathering this information. For example using the list_files tool is more effective than running a command like \`ls\` in the terminal. It's critical that you think about each available tool and use the one that best fits the current step in the task. | ||||||||||||||||||||||||||||||||||||||
| 3. If multiple actions are needed, you may use multiple tools in a single message when appropriate, or use tools iteratively across messages. Each tool use should be informed by the results of previous tool uses. Do not assume the outcome of any tool use. Each step must be informed by the previous step's result. | ||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||
| : "" | |
| return `# Tool Use Guidelines | |
| 1. Assess what information you already have and what information you need to proceed with the task. | |
| 2. Choose the most appropriate tool based on the task and the tool descriptions provided. Assess if you need additional information to proceed, and which of the available tools would be most effective for gathering this information. For example using the list_files tool is more effective than running a command like \`ls\` in the terminal. It's critical that you think about each available tool and use the one that best fits the current step in the task. | |
| 3. If multiple actions are needed, you may use multiple tools in a single message when appropriate, or use tools iteratively across messages. Each tool use should be informed by the results of previous tool uses. Do not assume the outcome of any tool use. Each step must be informed by the previous step's result. | |
| : ""; | |
| const step3Guideline = useXmlToolCalling | |
| ? `3. If multiple actions are needed, use tools iteratively across messages, making at most one XML tool call per assistant message. Each tool use should be informed by the results of previous tool uses. Do not assume the outcome of any tool use. Each step must be informed by the previous step's result.` | |
| : `3. If multiple actions are needed, you may use multiple tools in a single message when appropriate, or use tools iteratively across messages. Each tool use should be informed by the results of previous tool uses. Do not assume the outcome of any tool use. Each step must be informed by the previous step's result.`; | |
| return `# Tool Use Guidelines | |
| 1. Assess what information you already have and what information you need to proceed with the task. | |
| 2. Choose the most appropriate tool based on the task and the tool descriptions provided. Assess if you need additional information to proceed, and which of the available tools would be most effective for gathering this information. For example using the list_files tool is more effective than running a command like \`ls\` in the terminal. It's critical that you think about each available tool and use the one that best fits the current step in the task. | |
| ${step3Guideline} |
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -1,4 +1,52 @@ | ||||||
| export function getSharedToolUseSection(): string { | ||||||
| export function getSharedToolUseSection(useXmlToolCalling?: boolean): string { | ||||||
| if (useXmlToolCalling) { | ||||||
| return `==== | ||||||
|
|
||||||
| TOOL USE | ||||||
|
|
||||||
| You have access to a set of tools that are executed upon the user's approval. You can use one tool per message, and will receive the result of that tool use in the user's response. You use tools step-by-step to accomplish a given task, with each tool use informed by the result of the previous tool use. | ||||||
|
||||||
| You have access to a set of tools that are executed upon the user's approval. You can use one tool per message, and will receive the result of that tool use in the user's response. You use tools step-by-step to accomplish a given task, with each tool use informed by the result of the previous tool use. | |
| You have access to a set of tools that are executed upon the user's approval. You can use one or more tools per message, and will receive the result of those tool uses in the user's response. You use tools step-by-step to accomplish a given task, with each tool use informed by the result of the previous tool use. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The doc comment claims XML tool calls are parsed by TagMatcher in
presentAssistantMessage(), butpresentAssistantMessagecurrently treats missingtool_use.idas an invalid legacy/XML tool call and rejects it, and tools generally requirenativeArgs. Please update this comment to reflect the actual execution/parsing flow, or add the missing XML parsing implementation and adjust this description accordingly.