If I don't have enough engineering resources to implement both format validation and content-truthfulness verification simultaneously, which should be prioritized?
If you genuinely can only pick one to do first, prioritize format validation, but be clearly aware this only solves part of the problem — don't assume reliability engineering is complete just because format validation is done.
Why prioritize format validation: format validation's cost-benefit ratio is usually higher — implementation cost is relatively low (most mainstream LLM APIs already have Structured Output support built in, no need to build this mechanism from scratch yourself), but it immediately eliminates the most basic, most easily occurring failure mode of 'downstream code crashing from a parsing error.' If this failure isn't handled, the system might not even achieve basic stable operation, let alone get to discussing content truthfulness.
Once format validation is done, the next priority judgment: with limited resources, you don't need full content-truthfulness verification on every output field simultaneously — you can prioritize by 'how much actual damage would result if this field were wrong.' Fields involving money operations or irreversible actions (transaction amount, target address) get priority content-verification resources; fields purely for display, analysis reference, not directly triggering any actual action (an explanatory text summary) can have lower content-verification priority, since even slightly inaccurate content in that field usually causes limited actual damage.
A low-cost, high-value content-verification starting move: you don't need to build a complex verification system from the start. A low-cost but decent-value approach is requiring the model to attach a brief reasoning ground alongside outputting a key conclusion (as mentioned earlier, 'what data was cited, what logic it's based on') — even without further automated cross-verification, simply requiring the model to 'state its grounds' filters out a portion of especially low-quality output where the model itself can't articulate a concrete basis. This is a low-investment-cost starting move that filters out the most obvious problems, worth prioritizing under limited resources.
If I require the model to attach reasoning grounds, does that mean this output's content is definitely more trustworthy? Could the model also 'fabricate' a plausible-looking but actually fake reasoning process?
This is a very worthwhile question to raise, and the answer is: the model genuinely can fabricate a plausible-looking reasoning process that doesn't actually correspond to real grounds. Requiring the model to attach grounds can't guarantee those grounds are genuinely true, but it still has real value, for the following reasons.
The value of requiring attached grounds isn't 'guaranteeing the grounds are definitely true' — it's 'making the grounds inspectable.' If the model only outputs an isolated conclusion (e.g., 'this protocol's risk is low, recommend investing funds') with no grounds at all, a human reviewer or another verification Agent has absolutely no entry point to question or verify this conclusion — the conclusion itself is a black box. If the model is required to attach grounds (e.g., 'because this protocol's TVL has grown steadily over the past 90 days, and the audit report found no high-risk vulnerabilities'), even though these grounds could themselves be fabricated, they at least provide a concrete, verifiable entry point — a human reviewer or verification Agent can go confirm whether the specific statement 'this protocol's TVL genuinely grew steadily over the past 90 days' is true; if verification finds this specific statement itself is false, the entire conclusion can be overturned; without this specific statement, there's nowhere to even start checking.
A further defense: make the specific statements within the grounds as automatically verifiable as possible. Rather than just requiring the model to 'state its grounds,' a more rigorous design requires the grounds to include specific information that can be automatically cross-checked (specific numbers, specific citation sources, specific timestamps), rather than accepting a vague, unverifiable description like 'this protocol seems fairly robust.' The earlier article's mention of 'adding an extra verification step for independently verifiable fields' can extend to apply to the grounds field itself — if the grounds cite a specific TVL number, a downstream system can automatically query whether this number matches real on-chain data. This goes a step further than just requiring the model to 'explain its reasoning,' bringing 'are the grounds themselves true' into the scope of automated verification too.
Core mindset: requiring attached grounds transforms the hard-to-verify problem of 'content truthfulness' into the relatively easier-to-verify sub-problem of 'are the specific statements within the grounds true' — it's an engineering technique to lower verification difficulty, not a silver bullet that solves truthfulness once and for all. Fabricated reasoning grounds are still a possible occurrence, but compared to an isolated conclusion with no grounds at all, a conclusion with concrete, verifiable grounds makes the overall credibility-verification process go far more smoothly.
In an onchain Agent scenario, if the Agent's output involves a risk assessment of a DeFi protocol — a judgment that's fairly subjective with no absolute objective answer — how should a verification mechanism be designed?
A fairly subjective judgment genuinely can't use a binary right-or-wrong verification approach like 'checking whether a specific number is correct,' but there are still several viable verification directions, with the core idea being converting 'is this judgment absolutely correct' into 'does this judgment's reasoning process hold up.'
Direction one: decompose into multiple independently verifiable sub-judgments. A vague conclusion of 'this protocol's risk is low' is hard to verify directly, but you can require the model to decompose this conclusion into several more concrete sub-judgments — 'has the smart contract been audited by a reputable firm,' 'TVL scale and growth trend,' 'is the governance mechanism decentralized,' 'have there been past security incidents.' Each of these sub-judgments, while still carrying some subjective assessment component itself, is more easily independently verifiable than the vague 'lower risk' conclusion (whether it's 'been audited' can be directly checked — whether an audit report exists, whether it genuinely comes from the claimed auditing firm). The overall risk assessment's credibility can, to some extent, be indirectly assessed by verifying how solid multiple sub-judgments are, rather than directly verifying that vague final conclusion.
Direction two: consistency checking rather than absolute-correctness checking. Since there's no absolutely objective answer, a practically effective alternative verification method is 'consistency checking' — having the same Agent's multiple judgments on similar situations checked for whether the judgment logic stays consistent (if an Agent, facing two protocols with highly similar TVL, audit status, and governance mechanism, gives drastically different risk-assessment conclusions, this inconsistency itself is a signal worth alertness, indicating the judgment logic might not be stable enough, or was influenced by factors that shouldn't affect the judgment). Consistency alone can't prove 'the judgment is correct,' but lack of consistency is usually a strong signal that judgment quality has a problem.
Direction three: cross-checking multiple independent sources. For subjective judgment, a single Agent's conclusion has limited credibility — you can design it so multiple independent verification paths (another independent Agent, or a human reviewer) each do a risk assessment on the same protocol; if multiple independent sources' conclusions align highly, overall credibility improves; if a clear divergence appears, this indicates the judgment itself may involve controversy or uncertainty and should be flagged, rather than blindly adopting a single Agent's conclusion as the final word.
Core principle: for fairly subjective judgment, the verification goal isn't pursuing 'absolute correctness' (which may not exist to begin with) — it's pursuing 'a transparent, consistent judgment process with multiple independent perspectives corroborating each other.' This is an engineering practice for reducing uncertainty, not a silver bullet for eliminating it — ultimately, for high-risk subjective judgment, room for a human to make the final decision should still be preserved; the Agent provides supplementary reference information for judgment, not final authority replacing human judgment.
In a multi-agent system, if content generated by Sub-agent A (with attached reasoning grounds) is handed off to Sub-agent B, and B verifies this content and judges it trustworthy, does this 'verified' status need to be re-verified again when it later reaches the Orchestrator or other Sub-agents?
This question extends the 'pre-execution re-verification' principle discussed in the earlier Handoff entry, but the scenario here is more nuanced — if B has already done the verification, can subsequent steps simply trust the 'verified' status without repeating verification?
Core judgment principle: trust the 'verification result' itself, but confirm the verification's validity period and scope, rather than re-verifying entirely from scratch. B already spent resources doing the verification; if every subsequent step has to redo a full verification each time, that creates unnecessary duplicate cost, not an efficiency-reasonable design. A more reasonable approach: after B completes verification, treat the 'verification result' itself as a new, trustworthy field, appended to the handoff package and passed further downstream (e.g., adding fields like verification_status: verified, verified_by: sub_agent_B, verification_timestamp, and verification_scope — specifically which fields were verified, not a vague statement of verifying 'everything'). A downstream Orchestrator or other Sub-agent can directly trust this 'verified' status without re-executing the verification logic.
But this trust isn't unconditional — two things need checking. Whether the verification's validity period is still within a reasonable range: the earlier slippage-tolerance article discussed a similar concept — for time-sensitivity-critical information like market data and on-chain state, a 'verified' status carries an implicit expiration; if a long time passes between B completing verification and the Orchestrator actually using this data (exceeding a reasonable threshold, with the specific threshold depending on how fast the underlying data changes), even if the status field says 'verified,' this verification may have already gone stale and can't be trusted unconditionally — it should be treated as a case needing re-verification. Whether the verification's scope covers the part downstream actually needs: B's verification might have only checked certain specific fields within this content (e.g., only verifying the TVL number, not verifying the audit report's authenticity); if what downstream needs to use happens to be a field B didn't verify, the vague 'verified' flag can actually be misleading — meaning the verification-status record can't just be a vague boolean, and should, as mentioned, explicitly record the 'verification scope,' letting downstream accurately judge 'is the part I need to use actually within what's been verified.'
The core principle this design echoes: the verification result itself can and should be trusted and passed along, without needing every step to redo a full verification (this is the efficient design that properly makes use of work already completed upstream) — but the premise for trusting a verification result is that its expiration and scope must be explicitly recorded and checkable, rather than treating 'verified' as a vague, universal no-verification-needed pass.
Most Agent developers, when a system first launches, focus first on 'is the output format correct' — missing fields, JSON syntax errors, whether schema validation passes. This is a reasonable first step and a relatively easy problem to solve (through mechanisms like Structured Output, format error rates can be pushed close to zero). But once format reliability is solved, a more concealed, harder-to-notice problem truly surfaces: an output that's entirely correct in format can still be wrong in content — and this kind of error is harder to discover than a format error, because it doesn't trigger any obvious system alert.
What this article breaks down is exactly this — format reliability and content truthfulness, two problems frequently conflated yet entirely independent — and why most Agent system reliability engineering ends up accidentally pouring most of its effort into the former, while barely touching the latter.
What format validation (like Structured Output's schema constraints) can guarantee is, at its core, exactly one thing: the model's output data structure matches a predefined shape. A `confidence_score` field is a float between 0 and 1, a `source_url` field is a legally formatted URL string, a `status` field is one of a few predefined options — these are all within format validation's ability to check and intercept violating output.
What format validation entirely cannot check: whether this confidence score genuinely reflects the model's internal uncertainty, or the model just arbitrarily generated a plausible-looking number; whether this URL actually exists and its content genuinely matches what's cited, or the model conjured up a format-correct but non-existent link out of thin air; whether this status option accurately describes what actually happened, or the model picked a 'seemingly safer' option to dodge a genuine judgment. All of these questions fall entirely outside format validation's scope of ability, requiring an entirely different mechanism to address.
Format errors trigger an explicit system response — JSON parsing failure, schema validation not passing — these are all events that can be automatically detected and automatically alerted on; developers can easily see a 'format error rate' metric on a dashboard and optimize against it. Content errors trigger no automated alert whatsoever. An Agent generating a perfectly formatted but hallucinated analysis report is, from the system's perspective, a 'successful' execution — no error logs, no anomaly alerts. The only way to discover the problem is for someone to actually go verify whether the report's content is true, and this kind of manual verification is costly, impossible to do for every single output — meaning most content-level errors persist undiscovered until some instance causes real damage (a wrong trading decision made based on incorrect information), only then getting traced back and found.
There's a counterintuitive phenomenon worth specifically pointing out here: mechanisms like Structured Output, while greatly improving system reliability, may also inadvertently make content errors harder for a human to catch, because a formatted output itself carries a visual implication of 'this went through systematic processing, should be trustworthy.' A free-text model output, if its content has an obvious logical leap or contradiction, is usually easier for a human reader to notice (since reading free text spreads the reader's attention across the whole passage); but a neatly formatted JSON object is something a human — especially a downstream system doing only programmatic processing without human review — is instead more likely to directly trust every field's value, because 'format correct' is easily psychologically misread as 'content also verified.' This is a hidden cost format reliability brings, worth specifically being aware of when designing a system.
Since format validation can't solve content truthfulness, the actual solution must layer on top of format validation, with several directions here. For fields that can be independently verified (URLs, numerical market data, cited specific protocol names), add an extra verification step downstream, genuinely checking whether that field's content is true, rather than assuming format-correct means content-correct — echoing the layered-verification approach mentioned in the earlier Structured Output entry. For fields not easily directly verifiable, involving reasoning or judgment (analysis conclusions, risk assessments), require the model to attach reasoning grounds in its output (which specific data was cited, what logic the conclusion was based on), letting subsequent human review or another independent verification Agent cross-check the conclusion's plausibility, rather than only seeing an isolated conclusion with no supporting grounds. For high-risk decisions (involving actual money operations), no matter how complete and trustworthy the format looks, the repeatedly discussed human-confirmation defense line should still be retained — don't skip this step just because the output format is correct.
If you're developing an Agent system, implementing Structured Output is only the first step of reliability engineering, not the endpoint — launching a format validation mechanism easily produces an illusion of 'output is already reliable enough,' but this only solves the format-layer problem. The real next step worth investing resources in is designing an independent verification mechanism for 'is the content itself true,' especially for high-risk, high-impact output fields. If you're a user or evaluator of an Agent product, seeing an Agent's output formatted neatly and professionally, don't trust the content itself just from that visual impression — format tidiness and content truthfulness are two entirely independent things, worth evaluating separately.