Exam CCAR-F Topic 1 Question 143 Discussion
Actual exam question for Anthropic's CCAR-F exam
Question #: 143
Topic #: 1
Question #: 143
Topic #: 1
You are building a structured data extraction system using Claude. The system extracts information from unstructured documents, validates the output using JavaScript Object Notation (JSON) schemas, and maintains high accuracy. It must handle edge cases gracefully and integrate with downstream systems.
Your system has been running for 3 weeks and human reviewers have corrected 847 extractions. Analysis reveals a recurring pattern: when recipes use informal measurements like "a handful" or "a splash," the model either invents specific amounts or leaves fields empty-accounting for 23% of all corrections.
How should you use this feedback to improve extraction accuracy?
Your system has been running for 3 weeks and human reviewers have corrected 847 extractions. Analysis reveals a recurring pattern: when recipes use informal measurements like "a handful" or "a splash," the model either invents specific amounts or leaves fields empty-accounting for 23% of all corrections.
How should you use this feedback to improve extraction accuracy?
Suggested Answer: B Vote an answer
The reviewer corrections have exposed a narrow and repeatable interpretation failure. The desired policy is clear: informal measurements are valid source values and must be preserved verbatim rather than normalized into invented quantities or treated as missing. This behavior can be communicated efficiently through targeted few-shot examples.
Anthropic recommends examples for demonstrating expected behavior and improving consistency. Examples can pair source phrases such as "a handful of spinach," "a splash of vinegar," and "a pinch of salt" with outputs that retain handful , splash , and pinch exactly. Additional counterexamples can show that Claude must not convert these phrases into grams, millilitres, or estimated serving quantities. ( https://docs.anthropic.
com/en/docs/about-claude/use-case-guides/ticket-routing )
Option A introduces a substantially heavier training workflow for a problem that can be addressed directly through the prompt. Option C creates a parallel extraction mechanism based on pattern matching; it will be brittle across linguistic variations and may populate a value without understanding its relationship to the correct ingredient. Option D adds useful classification metadata, but it does not instruct Claude to preserve the original measurement instead of inventing or omitting it.
The revised prompt should combine an explicit verbatim-extraction rule with several varied examples derived from the corrected cases, followed by regression evaluation against the identified failure set.
Official references/topics: Few-Shot Examples, Feedback-Driven Prompt Improvement, Verbatim Extraction, Regression Evaluation.
Anthropic recommends examples for demonstrating expected behavior and improving consistency. Examples can pair source phrases such as "a handful of spinach," "a splash of vinegar," and "a pinch of salt" with outputs that retain handful , splash , and pinch exactly. Additional counterexamples can show that Claude must not convert these phrases into grams, millilitres, or estimated serving quantities. ( https://docs.anthropic.
com/en/docs/about-claude/use-case-guides/ticket-routing )
Option A introduces a substantially heavier training workflow for a problem that can be addressed directly through the prompt. Option C creates a parallel extraction mechanism based on pattern matching; it will be brittle across linguistic variations and may populate a value without understanding its relationship to the correct ingredient. Option D adds useful classification metadata, but it does not instruct Claude to preserve the original measurement instead of inventing or omitting it.
The revised prompt should combine an explicit verbatim-extraction rule with several varied examples derived from the corrected cases, followed by regression evaluation against the identified failure set.
Official references/topics: Few-Shot Examples, Feedback-Driven Prompt Improvement, Verbatim Extraction, Regression Evaluation.
by Ed at Sep 02, 2026, 10:03 PM
0
0
0
10
Comments
Upvoting a comment with a selected answer will also increase the vote count towards that answer by one. So if you see a comment that you already agree with, you can upvote it instead of posting a new comment.
Report Comment
Commenting
You can sign-up / login (it's free).