Enrich claim analysis and review support
This commit is contained in:
parent
a54082141a
commit
2f7696c115
|
|
@ -0,0 +1,172 @@
|
|||
# Evidence-Docket Claims Analysis
|
||||
|
||||
GroundRecall’s current import/review model is good at:
|
||||
|
||||
- preserving provenance
|
||||
- turning observations into reviewable claims
|
||||
- keeping concepts, claims, relations, and citations separate
|
||||
|
||||
It is still weak at a different task:
|
||||
|
||||
- structured adversarial or forensic analysis of an argument across multiple claim lanes
|
||||
|
||||
That gap became clearer in the local `evolutionnews.net` evidence-docket work.
|
||||
Those dockets do not just collect claims. They classify how claims function in
|
||||
an argument.
|
||||
|
||||
## Why this matters
|
||||
|
||||
In the Mason design-biology work, the useful analysis was not just:
|
||||
|
||||
- what claim appears in the text
|
||||
- what source supports it
|
||||
|
||||
It also depended on:
|
||||
|
||||
- what role the claim plays in the overall argument
|
||||
- whether the burden of proof is being shifted
|
||||
- whether multiple domains are being bundled rhetorically
|
||||
- whether citations merely exist or actually support the claim
|
||||
- what empirical gap is being asserted versus what research program already exists
|
||||
|
||||
GroundRecall can already hold the raw ingredients for that kind of work, but it
|
||||
does not yet model them explicitly.
|
||||
|
||||
## Evidence-docket structure worth borrowing
|
||||
|
||||
The local evidence-docket workflow has a few recurring sections that map well
|
||||
onto richer GroundRecall review:
|
||||
|
||||
1. `claim map`
|
||||
The operative argument structure, not just isolated statements.
|
||||
|
||||
2. `primary findings`
|
||||
Higher-order judgments such as burden asymmetry, domain bundling, or model
|
||||
overreach.
|
||||
|
||||
3. `evidence cards`
|
||||
Focused support packets that connect one objection or claim to a bounded
|
||||
source trail.
|
||||
|
||||
4. `rhetorical maneuvers flagged`
|
||||
Moves such as burden shift, equivocation, present-function fallacy, or
|
||||
overgeneralization from a narrow model.
|
||||
|
||||
5. `burden check`
|
||||
What the author requires from the opposing view versus what their own view
|
||||
must supply.
|
||||
|
||||
6. `citation and source audit`
|
||||
Whether named sources are real, relevant, overextended, or contradicted by
|
||||
the way they are being used.
|
||||
|
||||
7. `research program`
|
||||
What empirical work would actually reduce the leverage of the objection.
|
||||
|
||||
## Implications for GroundRecall
|
||||
|
||||
GroundRecall should stay centered on grounded records, but claim analysis can be
|
||||
enriched in a way that matches the docket workflow.
|
||||
|
||||
### 1. Expand claim kinds
|
||||
|
||||
Current `claim_kind` values are mostly low-level:
|
||||
|
||||
- `statement`
|
||||
- `summary`
|
||||
- adapter-specific kinds such as `mastery_signal`
|
||||
|
||||
Useful additions:
|
||||
|
||||
- `argument_step`
|
||||
- `burden_check`
|
||||
- `rhetorical_move`
|
||||
- `citation_audit`
|
||||
- `research_gap`
|
||||
- `research_program`
|
||||
- `counterexample`
|
||||
|
||||
These do not replace ordinary claims. They make higher-order analytical claims
|
||||
first-class instead of burying them in reviewer notes.
|
||||
|
||||
### 2. Add argument-lane metadata
|
||||
|
||||
Claims should be able to carry lightweight analytical tags such as:
|
||||
|
||||
- `argument_role`: premise, inference, objection, counterargument, scope note
|
||||
- `analysis_lane`: empirical, rhetorical, citation, burden, research_program
|
||||
- `risk_flags`: overstatement, bundling, equivocation, unsupported_generalization
|
||||
|
||||
This can start as claim metadata without requiring a schema break.
|
||||
|
||||
### 3. Model evidence cards explicitly
|
||||
|
||||
An evidence card is more than one claim. It is a bounded support packet that
|
||||
ties together:
|
||||
|
||||
- one focal issue
|
||||
- one or more claims
|
||||
- supporting observations
|
||||
- cited sources
|
||||
- reviewer verdict
|
||||
|
||||
GroundRecall does not need a new top-level store object immediately. A first
|
||||
step could be review-export grouping by:
|
||||
|
||||
- lane
|
||||
- concept
|
||||
- citation cluster
|
||||
|
||||
## Bibliography and abstracts
|
||||
|
||||
The bibliography expansion work showed that abstracts are often the fastest way
|
||||
to estimate:
|
||||
|
||||
- whether a source is in the right domain
|
||||
- whether it actually addresses the asserted mechanism or phenomenon
|
||||
- whether a citation is likely to support or overstate a claim
|
||||
|
||||
That suggests two concrete upgrades for GroundRecall review:
|
||||
|
||||
1. show more than citation-key existence
|
||||
Review should expose whether resolved bibliography entries have abstracts,
|
||||
DOI coverage, and enough metadata depth for meaningful support judgment.
|
||||
|
||||
2. use abstracts as first-pass support context
|
||||
Abstract snippets should be available when a reviewer is deciding whether a
|
||||
cited work materially supports a claim or merely sounds adjacent.
|
||||
|
||||
Important boundary:
|
||||
|
||||
- abstracts are triage evidence, not final adjudication
|
||||
- direct source reading still matters for strong or controversial claims
|
||||
|
||||
## Recommended implementation order
|
||||
|
||||
1. Enrich bibliography summary and artifact citation summaries.
|
||||
Surface abstract-bearing coverage, representative titles, DOI coverage, and
|
||||
short abstract snippets in review payloads.
|
||||
|
||||
2. Add analytical claim metadata.
|
||||
Start with optional metadata fields in claim rows and review exports.
|
||||
|
||||
3. Add review lanes mirroring the evidence-docket workflow.
|
||||
Separate empirical support review from rhetorical and burden-check review.
|
||||
|
||||
4. Add evidence-card grouping in review UI/export.
|
||||
Let reviewers inspect a bounded packet instead of isolated claim rows.
|
||||
|
||||
5. Add a bibliography-assisted claim-support pass.
|
||||
Reuse CiteGeist support/verification capabilities so GroundRecall can move
|
||||
from “citation exists” toward “citation probably supports this claim because…”
|
||||
|
||||
## Practical near-term change
|
||||
|
||||
The smallest worthwhile next step is:
|
||||
|
||||
- improve GroundRecall review payloads so bibliography strength is visible
|
||||
- especially abstract-bearing resolved entries and representative titles
|
||||
|
||||
That does not solve richer claim analysis by itself, but it gives reviewers a
|
||||
better support surface and aligns GroundRecall with the successful parts of the
|
||||
evidence-docket workflow.
|
||||
|
|
@ -122,10 +122,30 @@ def materialize_citegeist_store(import_dir: str | Path, source_root: str | Path)
|
|||
def bibliography_summary_payload(source_root: str | Path) -> dict[str, Any]:
|
||||
index = load_bibliography_index(source_root)
|
||||
source_files = discover_bib_files(source_root)
|
||||
abstract_entry_count = 0
|
||||
doi_entry_count = 0
|
||||
years: list[int] = []
|
||||
representative_titles: list[str] = []
|
||||
for payload in index.values():
|
||||
fields = payload.get("fields", {})
|
||||
if str(fields.get("abstract", "")).strip():
|
||||
abstract_entry_count += 1
|
||||
if str(fields.get("doi", "")).strip():
|
||||
doi_entry_count += 1
|
||||
year_text = str(fields.get("year", "")).strip()
|
||||
if year_text.isdigit():
|
||||
years.append(int(year_text))
|
||||
title = str(fields.get("title", "")).strip()
|
||||
if title and len(representative_titles) < 5:
|
||||
representative_titles.append(title)
|
||||
return {
|
||||
"enabled": bool(index),
|
||||
"entry_count": len(index),
|
||||
"source_files": [str(path.relative_to(Path(source_root))) for path in source_files],
|
||||
"abstract_entry_count": abstract_entry_count,
|
||||
"doi_entry_count": doi_entry_count,
|
||||
"year_range": [min(years), max(years)] if years else [],
|
||||
"representative_titles": representative_titles,
|
||||
}
|
||||
|
||||
|
||||
|
|
@ -151,6 +171,80 @@ def serialize_citegeist_entry_payload(payload: dict[str, Any] | None) -> dict[st
|
|||
return json.loads(json.dumps(result))
|
||||
|
||||
|
||||
_SUPPORT_TOKEN_RE = re.compile(r"[a-z0-9]{4,}")
|
||||
|
||||
|
||||
def build_local_claim_support_suggestions(
|
||||
bibliography_index: dict[str, dict[str, Any]],
|
||||
claim_text: str,
|
||||
*,
|
||||
context: str = "",
|
||||
limit: int = 3,
|
||||
exclude_keys: set[str] | None = None,
|
||||
) -> list[dict[str, Any]]:
|
||||
claim_tokens = _support_tokens(claim_text)
|
||||
context_tokens = _support_tokens(context)
|
||||
combined_tokens = claim_tokens | context_tokens
|
||||
exclude = {item for item in (exclude_keys or set()) if item}
|
||||
scored: list[tuple[float, dict[str, Any]]] = []
|
||||
|
||||
for citation_key, payload in bibliography_index.items():
|
||||
if citation_key in exclude:
|
||||
continue
|
||||
fields = payload.get("fields", {})
|
||||
title = str(fields.get("title", "")).strip()
|
||||
abstract = str(fields.get("abstract", "")).strip()
|
||||
venue = str(fields.get("journal", "") or fields.get("booktitle", "") or fields.get("publisher", "")).strip()
|
||||
doi = str(fields.get("doi", "")).strip()
|
||||
haystack_tokens = _support_tokens(" ".join(part for part in (title, abstract, venue) if part))
|
||||
if not haystack_tokens:
|
||||
continue
|
||||
overlap = combined_tokens & haystack_tokens
|
||||
if not overlap:
|
||||
continue
|
||||
title_tokens = _support_tokens(title)
|
||||
abstract_tokens = _support_tokens(abstract)
|
||||
title_overlap = claim_tokens & title_tokens
|
||||
abstract_overlap = claim_tokens & abstract_tokens
|
||||
score = (len(title_overlap) * 2.0) + len(abstract_overlap) + (0.5 if abstract else 0.0) + (0.25 if doi else 0.0)
|
||||
scored.append(
|
||||
(
|
||||
score,
|
||||
{
|
||||
"citation_key": citation_key,
|
||||
"title": title,
|
||||
"year": str(fields.get("year", "")).strip(),
|
||||
"authors": str(fields.get("author", "")).strip(),
|
||||
"venue": venue,
|
||||
"doi": doi,
|
||||
"score": round(score, 3),
|
||||
"reason": _support_reason(title_overlap, abstract_overlap, abstract=bool(abstract), context_overlap=bool(context_tokens & haystack_tokens)),
|
||||
"abstract_snippet": abstract.replace("\n", " ")[:280] if abstract else "",
|
||||
},
|
||||
)
|
||||
)
|
||||
|
||||
scored.sort(key=lambda item: (-item[0], item[1]["year"], item[1]["title"]))
|
||||
return [item[1] for item in scored[: max(0, limit)]]
|
||||
|
||||
|
||||
def _support_tokens(text: str) -> set[str]:
|
||||
return {match.group(0) for match in _SUPPORT_TOKEN_RE.finditer(text.lower())}
|
||||
|
||||
|
||||
def _support_reason(title_overlap: set[str], abstract_overlap: set[str], *, abstract: bool, context_overlap: bool) -> str:
|
||||
reasons: list[str] = []
|
||||
if title_overlap:
|
||||
reasons.append("title overlap")
|
||||
if abstract_overlap:
|
||||
reasons.append("abstract overlap")
|
||||
if context_overlap:
|
||||
reasons.append("context overlap")
|
||||
if abstract and not abstract_overlap:
|
||||
reasons.append("abstract available")
|
||||
return ", ".join(reasons) if reasons else "local bibliography match"
|
||||
|
||||
|
||||
def _parse_bib_entries(text: str, *, symbols: dict[str, Any] | None) -> list[Any]:
|
||||
if symbols is not None:
|
||||
try:
|
||||
|
|
|
|||
|
|
@ -101,11 +101,25 @@ def build_claim_record(
|
|||
index: int,
|
||||
fragment_ids: list[str] | None = None,
|
||||
) -> dict[str, Any]:
|
||||
claim_kind = "statement" if observation_record["role"] == "claim" else "summary"
|
||||
argument_role = "premise" if claim_kind == "statement" else "context"
|
||||
risk_flags: list[str] = []
|
||||
if observation.contradict_keys:
|
||||
argument_role = "counterargument"
|
||||
risk_flags.append("contradiction_linked")
|
||||
if observation.supersede_keys:
|
||||
argument_role = "revision"
|
||||
risk_flags.append("supersession_linked")
|
||||
return {
|
||||
"claim_id": _claim_id_for_observation(observation_record, observation, index),
|
||||
"import_id": context.import_id,
|
||||
"claim_text": observation_record["text"],
|
||||
"claim_kind": "statement" if observation_record["role"] == "claim" else "summary",
|
||||
"claim_kind": claim_kind,
|
||||
"metadata": {
|
||||
"analysis_lane": "empirical",
|
||||
"argument_role": argument_role,
|
||||
"risk_flags": risk_flags,
|
||||
},
|
||||
"source_observation_ids": [observation_record["observation_id"]],
|
||||
"supporting_fragment_ids": list(fragment_ids or []),
|
||||
"concept_ids": [f"concept::{concept_id}" for concept_id in concept_ids],
|
||||
|
|
|
|||
|
|
@ -69,6 +69,7 @@ class ClaimRecord(BaseModel):
|
|||
claim_id: str
|
||||
claim_text: str
|
||||
claim_kind: str = "statement"
|
||||
metadata: dict = Field(default_factory=dict)
|
||||
source_observation_ids: list[str] = Field(default_factory=list)
|
||||
supporting_fragment_ids: list[str] = Field(default_factory=list)
|
||||
concept_ids: list[str] = Field(default_factory=list)
|
||||
|
|
|
|||
|
|
@ -209,6 +209,7 @@ def promote_import_to_store(
|
|||
claim_id=claim["claim_id"],
|
||||
claim_text=claim.get("claim_text", ""),
|
||||
claim_kind=claim.get("claim_kind", "statement"),
|
||||
metadata=dict(claim.get("metadata", {})),
|
||||
source_observation_ids=list(claim.get("source_observation_ids", [])),
|
||||
supporting_fragment_ids=list(claim.get("supporting_fragment_ids", [])),
|
||||
concept_ids=concept_ids,
|
||||
|
|
|
|||
|
|
@ -126,12 +126,21 @@ function renderConceptPanel(concept) {
|
|||
const review = concept.review || {};
|
||||
const statusSpec = (state.reviewData.field_specs || []).find((item) => item.field === "status");
|
||||
const guidance = (state.reviewData.review_guidance?.priorities || []).map((item) => `<li>${escapeHtml(item)}</li>`).join("");
|
||||
const laneGuidance = (state.reviewData.review_guidance?.analysis_lanes || []).map((item) => `<li>${escapeHtml(item)}</li>`).join("");
|
||||
const laneSummary = Object.entries(review.analysis_lanes || {}).map(([lane, count]) => `
|
||||
<span class="chip">${escapeHtml(lane)} · ${escapeHtml(count)}</span>
|
||||
`).join("");
|
||||
const claims = (review.top_claims || []).map((claim) => `
|
||||
<article class="claim-card">
|
||||
<div class="claim-head">
|
||||
<strong>${escapeHtml(claim.claim_kind || "claim")}</strong>
|
||||
<span class="chip">${escapeHtml(claim.grounding_status || "unknown")}</span>
|
||||
</div>
|
||||
<div class="meta-row">
|
||||
<span class="chip">${escapeHtml(claim.analysis_lane || "empirical")}</span>
|
||||
<span class="chip">${escapeHtml(claim.argument_role || "premise")}</span>
|
||||
${(claim.risk_flags || []).map((flag) => `<span class="chip chip-warn">${escapeHtml(flag)}</span>`).join("")}
|
||||
</div>
|
||||
<p>${escapeHtml(claim.claim_text || "")}</p>
|
||||
<div class="tiny">Artifacts: ${escapeHtml((claim.artifact_paths || []).join(", ") || "none")}</div>
|
||||
${(claim.supporting_observations || []).slice(0, 2).map((obs) => `
|
||||
|
|
@ -140,6 +149,19 @@ function renderConceptPanel(concept) {
|
|||
<div>${escapeHtml(obs.text || "")}</div>
|
||||
</div>
|
||||
`).join("")}
|
||||
${(claim.support_suggestions || []).length ? `
|
||||
<div class="suggestion-block">
|
||||
<div class="tiny"><strong>Local support suggestions</strong></div>
|
||||
${(claim.support_suggestions || []).map((item) => `
|
||||
<div class="support-block">
|
||||
<div><strong>${escapeHtml(item.title || item.citation_key || "candidate source")}</strong></div>
|
||||
<div class="tiny">${escapeHtml(item.citation_key || "")}${item.year ? ` · ${escapeHtml(item.year)}` : ""}${item.venue ? ` · ${escapeHtml(item.venue)}` : ""}</div>
|
||||
<div class="tiny">${escapeHtml(item.reason || "")}${item.score !== undefined ? ` · score ${escapeHtml(item.score)}` : ""}</div>
|
||||
${item.abstract_snippet ? `<div>${escapeHtml(item.abstract_snippet)}</div>` : ""}
|
||||
</div>
|
||||
`).join("")}
|
||||
</div>
|
||||
` : ""}
|
||||
</article>
|
||||
`).join("");
|
||||
|
||||
|
|
@ -179,6 +201,11 @@ function renderConceptPanel(concept) {
|
|||
<h3>Reviewer guidance</h3>
|
||||
<ul>${guidance}</ul>
|
||||
</section>
|
||||
<section class="subpanel">
|
||||
<h3>Analysis lanes</h3>
|
||||
<div class="chip-row">${laneSummary || "<span class=\"muted\">No analytical lane summary available.</span>"}</div>
|
||||
<ul>${laneGuidance}</ul>
|
||||
</section>
|
||||
<section class="subpanel">
|
||||
<h3>Representative claims</h3>
|
||||
<div class="stack">${claims || "<div class=\"muted\">No representative claims available.</div>"}</div>
|
||||
|
|
|
|||
|
|
@ -194,6 +194,12 @@ textarea {
|
|||
border: 1px solid var(--line);
|
||||
}
|
||||
|
||||
.suggestion-block {
|
||||
margin-top: 12px;
|
||||
display: grid;
|
||||
gap: 10px;
|
||||
}
|
||||
|
||||
.claim-head {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
|
|
@ -201,6 +207,14 @@ textarea {
|
|||
margin-bottom: 8px;
|
||||
}
|
||||
|
||||
.meta-row,
|
||||
.chip-row {
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
gap: 8px;
|
||||
margin-bottom: 8px;
|
||||
}
|
||||
|
||||
.chip,
|
||||
.pill {
|
||||
display: inline-flex;
|
||||
|
|
@ -220,6 +234,10 @@ textarea {
|
|||
color: var(--warn);
|
||||
}
|
||||
|
||||
.chip-warn {
|
||||
color: var(--warn);
|
||||
}
|
||||
|
||||
ul {
|
||||
margin: 0;
|
||||
padding-left: 20px;
|
||||
|
|
|
|||
|
|
@ -6,7 +6,12 @@ import re
|
|||
import sys
|
||||
from collections import defaultdict
|
||||
from typing import Any, Callable
|
||||
from .citation_support import bibliography_summary_payload, load_bibliography_index, serialize_bib_entry
|
||||
from .citation_support import (
|
||||
bibliography_summary_payload,
|
||||
build_local_claim_support_suggestions,
|
||||
load_bibliography_index,
|
||||
serialize_bib_entry,
|
||||
)
|
||||
from .review_schema import CitationReviewEntry, ReviewSession
|
||||
|
||||
def export_review_state_json(session: ReviewSession, path: str | Path) -> None:
|
||||
|
|
@ -220,15 +225,57 @@ def _artifact_citation_payloads(
|
|||
"extracted_reference_count": len(extracted_refs),
|
||||
"citegeist_backends": backends,
|
||||
}
|
||||
resolved_entries = [entry for entry in payload["resolved_entries"] if entry]
|
||||
abstract_entries = [
|
||||
entry
|
||||
for entry in resolved_entries
|
||||
if str(entry.get("fields", {}).get("abstract", "")).strip()
|
||||
]
|
||||
artifact_payloads.append(payload)
|
||||
summaries[artifact["artifact_id"]] = {
|
||||
"citation_key_count": len(citation_keys),
|
||||
"extracted_reference_count": len(extracted_refs),
|
||||
"resolved_entry_count": len(resolved_entries),
|
||||
"abstract_entry_count": len(abstract_entries),
|
||||
"title_samples": [
|
||||
str(entry.get("fields", {}).get("title", "")).strip()
|
||||
for entry in resolved_entries[:3]
|
||||
if str(entry.get("fields", {}).get("title", "")).strip()
|
||||
],
|
||||
"abstract_snippets": [
|
||||
str(entry.get("fields", {}).get("abstract", "")).strip().replace("\n", " ")[:280]
|
||||
for entry in abstract_entries[:2]
|
||||
],
|
||||
"has_citation_support": bool(citation_keys or extracted_refs),
|
||||
}
|
||||
return artifact_payloads, summaries
|
||||
|
||||
|
||||
def _claim_analysis_metadata(claim: dict[str, Any]) -> dict[str, Any]:
|
||||
metadata = dict(claim.get("metadata", {}))
|
||||
lane = str(metadata.get("analysis_lane", "")).strip() or "empirical"
|
||||
argument_role = str(metadata.get("argument_role", "")).strip()
|
||||
if not argument_role:
|
||||
if claim.get("contradicts_claim_ids"):
|
||||
argument_role = "counterargument"
|
||||
elif claim.get("supersedes_claim_ids"):
|
||||
argument_role = "revision"
|
||||
elif claim.get("claim_kind") == "summary":
|
||||
argument_role = "context"
|
||||
else:
|
||||
argument_role = "premise"
|
||||
risk_flags = [str(item) for item in metadata.get("risk_flags", []) if str(item).strip()]
|
||||
if claim.get("contradicts_claim_ids") and "contradiction_linked" not in risk_flags:
|
||||
risk_flags.append("contradiction_linked")
|
||||
if claim.get("supersedes_claim_ids") and "supersession_linked" not in risk_flags:
|
||||
risk_flags.append("supersession_linked")
|
||||
return {
|
||||
"analysis_lane": lane,
|
||||
"argument_role": argument_role,
|
||||
"risk_flags": risk_flags,
|
||||
}
|
||||
|
||||
|
||||
def build_citation_review_entries_from_import(import_dir: str | Path) -> list[CitationReviewEntry]:
|
||||
base = Path(import_dir)
|
||||
manifest = _read_json(base / "manifest.json")
|
||||
|
|
@ -310,6 +357,7 @@ def build_citation_review_entries_from_import(import_dir: str | Path) -> list[Ci
|
|||
def _build_import_review_payload(session: ReviewSession, import_dir: Path) -> dict[str, Any]:
|
||||
manifest = _read_json(import_dir / "manifest.json")
|
||||
resolved_source_root = _resolve_source_root(import_dir, manifest.get("source_root", ""))
|
||||
bibliography_index = load_bibliography_index(resolved_source_root) if resolved_source_root else {}
|
||||
lint_payload = _read_json(import_dir / "lint_findings.json")
|
||||
queue_payload = _read_json(import_dir / "review_queue.json")
|
||||
graph_payload = _read_json(import_dir / "graph_diagnostics.json")
|
||||
|
|
@ -344,16 +392,37 @@ def _build_import_review_payload(session: ReviewSession, import_dir: Path) -> di
|
|||
queue_entry = queue_by_candidate_id.get(full_concept_id, {})
|
||||
claim_payloads: list[dict[str, Any]] = []
|
||||
has_citation_support = False
|
||||
lane_counts: dict[str, int] = defaultdict(int)
|
||||
for claim in concept_claims[:25]:
|
||||
supporting_observations = [observations_by_id[item] for item in claim.get("source_observation_ids", []) if item in observations_by_id]
|
||||
artifact_ids = {item["artifact_id"] for item in supporting_observations}
|
||||
citation_support = [artifact_citation_summary.get(artifact_id, {}) for artifact_id in artifact_ids]
|
||||
has_citation_support = has_citation_support or any(item.get("has_citation_support") for item in citation_support)
|
||||
analysis = _claim_analysis_metadata(claim)
|
||||
lane_counts[analysis["analysis_lane"]] += 1
|
||||
cited_keys = {
|
||||
key
|
||||
for artifact_id in artifact_ids
|
||||
for key in next(
|
||||
(item.get("citation_keys", []) for item in artifact_citations if item.get("artifact_id") == artifact_id),
|
||||
[],
|
||||
)
|
||||
}
|
||||
support_suggestions = build_local_claim_support_suggestions(
|
||||
bibliography_index,
|
||||
claim.get("claim_text", ""),
|
||||
context=concept.title,
|
||||
limit=3,
|
||||
exclude_keys=cited_keys,
|
||||
)
|
||||
claim_payloads.append(
|
||||
{
|
||||
"claim_id": claim["claim_id"],
|
||||
"claim_text": claim.get("claim_text", ""),
|
||||
"claim_kind": claim.get("claim_kind", ""),
|
||||
"analysis_lane": analysis["analysis_lane"],
|
||||
"argument_role": analysis["argument_role"],
|
||||
"risk_flags": analysis["risk_flags"],
|
||||
"grounding_status": claim.get("grounding_status", "unknown"),
|
||||
"supporting_observations": [
|
||||
{
|
||||
|
|
@ -367,6 +436,7 @@ def _build_import_review_payload(session: ReviewSession, import_dir: Path) -> di
|
|||
for obs in supporting_observations
|
||||
],
|
||||
"citation_support": citation_support,
|
||||
"support_suggestions": support_suggestions,
|
||||
"artifact_paths": [artifact_by_id[item]["path"] for item in artifact_ids if item in artifact_by_id],
|
||||
"finding_messages": [item["message"] for item in findings_by_target.get(claim["claim_id"], [])],
|
||||
}
|
||||
|
|
@ -390,6 +460,7 @@ def _build_import_review_payload(session: ReviewSession, import_dir: Path) -> di
|
|||
"triage_lane": str(queue_entry.get("triage_lane", "knowledge_capture")),
|
||||
"finding_codes": list(queue_entry.get("finding_codes", [])),
|
||||
"graph_codes": list(queue_entry.get("graph_codes", [])),
|
||||
"analysis_lanes": dict(sorted(lane_counts.items())),
|
||||
"top_claims": claim_payloads,
|
||||
"notes": list(concept.notes),
|
||||
}
|
||||
|
|
@ -413,10 +484,18 @@ def _build_import_review_payload(session: ReviewSession, import_dir: Path) -> di
|
|||
"Downgrade or reject concepts whose claims are fragmented, duplicated, or missing meaningful support.",
|
||||
"For academic material, citation-bearing claims deserve special scrutiny for fit, contradiction, and fabrication risk.",
|
||||
],
|
||||
"analysis_lanes": [
|
||||
"Empirical lane: what the source directly supports.",
|
||||
"Citation lane: whether cited work exists and materially fits the claim.",
|
||||
"Burden lane: what explanatory burden is being imposed or evaded.",
|
||||
"Rhetorical lane: bundling, overstatement, equivocation, or burden shifting.",
|
||||
"Research-program lane: what evidence or experiments would reduce the objection.",
|
||||
],
|
||||
"citation_guidance": [
|
||||
"A citation key or extracted reference is evidence of traceability, not correctness.",
|
||||
"Check whether the cited work actually supports the claim and whether the claim overstates it.",
|
||||
"Use the citation track to prioritize claims that can move into a separate citation-ingestion workflow.",
|
||||
"Treat abstract-based support suggestions as triage help, not as a substitute for direct source inspection.",
|
||||
],
|
||||
},
|
||||
"field_specs": [
|
||||
|
|
|
|||
|
|
@ -55,6 +55,8 @@ def test_groundrecall_import_emits_normalized_artifacts(tmp_path: Path) -> None:
|
|||
claims = _read_jsonl(result.out_dir / "claims.jsonl")
|
||||
assert any("Reliable rate upper bound" in item["claim_text"] for item in claims)
|
||||
assert any(item["supporting_fragment_ids"] for item in claims)
|
||||
assert all("metadata" in item for item in claims)
|
||||
assert any(item["metadata"].get("analysis_lane") == "empirical" for item in claims)
|
||||
|
||||
concepts = _read_jsonl(result.out_dir / "concepts.jsonl")
|
||||
concept_ids = {item["concept_id"] for item in concepts}
|
||||
|
|
@ -87,6 +89,7 @@ def test_groundrecall_import_emits_normalized_artifacts(tmp_path: Path) -> None:
|
|||
assert "concept_reviews" in review_data
|
||||
assert "citations" in review_data
|
||||
assert "citation_reviews" in review_data
|
||||
assert "analysis_lanes" in review_data["review_guidance"]
|
||||
|
||||
|
||||
def test_concept_standardization_merges_duplicate_titles_into_aliases() -> None:
|
||||
|
|
|
|||
|
|
@ -79,7 +79,9 @@ def test_review_workspace_resolves_citation_metadata_from_bibtex(tmp_path: Path)
|
|||
" author = {W. M. Baum},\n"
|
||||
" title = {On two types of deviation from the matching law: Bias and undermatching},\n"
|
||||
" journal = {Journal of the Experimental Analysis of Behavior},\n"
|
||||
" year = {1974}\n"
|
||||
" year = {1974},\n"
|
||||
" doi = {10.1901/jeab.1974.22-231},\n"
|
||||
" abstract = {Classic analysis of deviations from the matching law in operant choice experiments.}\n"
|
||||
"}\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
|
@ -93,3 +95,47 @@ def test_review_workspace_resolves_citation_metadata_from_bibtex(tmp_path: Path)
|
|||
assert entry["source_bib_path"] == "refs.bib"
|
||||
assert entry["raw_bibtex"]
|
||||
assert payload["bibliography"]["entry_count"] >= 1
|
||||
assert payload["bibliography"]["abstract_entry_count"] == 1
|
||||
assert payload["bibliography"]["doi_entry_count"] == 1
|
||||
assert payload["bibliography"]["year_range"] == [1974, 1974]
|
||||
concept_review = next(item for item in payload["concept_reviews"] if item["concept_id"] == "matching")
|
||||
assert "analysis_lanes" in concept_review
|
||||
citation_support = concept_review["top_claims"][0]["citation_support"][0]
|
||||
assert concept_review["top_claims"][0]["analysis_lane"] == "empirical"
|
||||
assert concept_review["top_claims"][0]["argument_role"] in {"premise", "context"}
|
||||
assert citation_support["resolved_entry_count"] == 1
|
||||
assert citation_support["abstract_entry_count"] == 1
|
||||
assert "matching law" in citation_support["abstract_snippets"][0].lower()
|
||||
suggestions = concept_review["top_claims"][0]["support_suggestions"]
|
||||
assert suggestions == []
|
||||
|
||||
|
||||
def test_review_workspace_surfaces_local_bibliography_support_suggestions(tmp_path: Path) -> None:
|
||||
root = tmp_path / "llmwiki"
|
||||
(root / "wiki").mkdir(parents=True)
|
||||
(root / "wiki" / "drift.md").write_text(
|
||||
"# Drift\n\n"
|
||||
"- Random genetic drift can dominate allele-frequency change in small populations.\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
(root / "refs.bib").write_text(
|
||||
"@article{kimura1968evolutionary,\n"
|
||||
" author = {Motoo Kimura},\n"
|
||||
" title = {Evolutionary Rate at the Molecular Level},\n"
|
||||
" journal = {Nature},\n"
|
||||
" year = {1968},\n"
|
||||
" abstract = {The rate of molecular evolution is compatible with neutral changes driven by random genetic drift in populations.}\n"
|
||||
"}\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
import_result = run_groundrecall_import(root, out_root=tmp_path / "imports", mode="quick", import_id="support-suggestions")
|
||||
workspace = GroundRecallReviewWorkspace(import_result.out_dir)
|
||||
payload = workspace.load_review_data()
|
||||
|
||||
concept_review = next(item for item in payload["concept_reviews"] if item["concept_id"] == "drift")
|
||||
suggestions = concept_review["top_claims"][0]["support_suggestions"]
|
||||
assert concept_review["analysis_lanes"]["empirical"] >= 1
|
||||
assert suggestions
|
||||
assert suggestions[0]["citation_key"] == "kimura1968evolutionary"
|
||||
assert "abstract" in suggestions[0]["reason"].lower() or "title" in suggestions[0]["reason"].lower()
|
||||
|
|
|
|||
Loading…
Reference in New Issue