Compare commits

...

2 Commits

Author SHA1 Message Date
welsberr 8b4359f4cc Added course ingestion pipeline. 2026-03-13 06:22:39 -04:00
welsberr db2cca50d0 Added mastery ledger and capability export. 2026-03-13 05:49:26 -04:00
27 changed files with 1060 additions and 268 deletions

View File

@ -8,10 +8,101 @@
## Recent revisions ## Recent revisions
### Course Ingestion Pipeline
This revision adds a **Course-to-Pack Ingestion Pipeline** plus a **stable rule-policy adapter layer**.
The design goal is to turn open or user-supplied course materials into draft
Didactopus domain packs without introducing a brittle external rule-engine dependency.
#### Why no third-party rule engine here?
To minimize dependency risk, this scaffold uses a small declarative rule-policy
adapter implemented in pure Python and standard-library data structures.
That gives Didactopus:
- portable rules
- inspectable rule definitions
- deterministic behavior
- zero extra runtime dependency for policy evaluation
If a stronger rule engine is needed later, this adapter can remain the stable API surface.
#### What is included
- normalized course schema
- Markdown/HTML-ish text ingestion adapter
- module / lesson / objective extraction
- concept candidate extraction
- prerequisite guess generation
- rule-policy adapter
- draft pack emitter
- review report generation
- sample course input
- sample generated pack outputs
### Mastery Ledger
This revision adds a **Mastery Ledger + Capability Export** layer.
The main purpose is to let Didactopus turn accumulated learner state into
portable, inspectable artifacts that can support downstream deployment,
review, orchestration, or certification-like workflows.
#### What is new
- mastery ledger data model
- capability profile export
- JSON export of mastered concepts and evaluator summaries
- Markdown export of a readable capability report
- artifact manifest for produced deliverables
- demo CLI for generating exports for an AI student or human learner
- FAQ covering how learned mastery is represented and put to work
#### Why this matters
Didactopus can now do more than guide learning. It can also emit a structured
statement of what a learner appears able to do, based on explicit concepts,
evidence, and artifacts.
That makes it easier to use Didactopus as:
- a mastery tracker
- a portfolio generator
- a deployment-readiness aid
- an orchestration input for agent routing
#### Mastery representation
A learner's mastery is represented as structured operational state, including:
- mastered concepts
- evaluator results
- evidence summaries
- weak dimensions
- attempt history
- produced artifacts
- capability export
This is stricter than a normal chat transcript or self-description.
#### Future direction
A later revision should connect the capability export with:
- formal evaluator outputs
- signed evidence ledgers
- domain-specific capability schemas
- deployment policies for agent routing
### Evaluator Pipeline
This revision introduces a **pluggable evaluator pipeline** that converts This revision introduces a **pluggable evaluator pipeline** that converts
learner attempts into structured mastery evidence. learner attempts into structured mastery evidence.
The prior revision adds an **agentic learner loop** that turns Didactopus into a closed-loop mastery system prototype. ### Agentic Learner Loop
This revision adds an **agentic learner loop** that turns Didactopus into a closed-loop mastery system prototype.
The loop can now: The loop can now:
@ -90,3 +181,4 @@ didactopus/
└── tests/ └── tests/
``` ```

View File

@ -1,18 +1,11 @@
model_provider: course_ingest:
mode: local_first default_pack_author: "Wesley R. Elsberry"
local: default_license: "REVIEW-REQUIRED"
backend: ollama min_term_length: 4
endpoint: http://localhost:11434 max_terms_per_lesson: 8
model_name: llama3.1:8b
platform: rule_policy:
default_dimension_thresholds: enable_prerequisite_order_rule: true
correctness: 0.8 enable_duplicate_term_merge_rule: true
explanation: 0.75 enable_project_detection_rule: true
transfer: 0.7 enable_review_flags: true
project_execution: 0.75
critique: 0.7
artifacts:
local_pack_dirs:
- domain-packs

35
docs/course-to-pack.md Normal file
View File

@ -0,0 +1,35 @@
# Course-to-Pack Ingestion Pipeline
The course-to-pack pipeline transforms educational material into Didactopus-native artifacts.
## Inputs
Typical sources:
- syllabus text
- lesson outlines
- markdown notes
- HTML course pages
- assignment sheets
- quiz prompts
- lecture transcripts
## Normalized intermediate structure
The pipeline builds a `NormalizedCourse` object containing:
- title
- source metadata
- modules
- lessons
- learning objectives
- exercises
- key terms
- project prompts
## Rule-policy adapter
The pipeline includes a small rule layer for stable policy transforms such as:
- suggest prerequisites from ordering
- merge repeated key-term candidates
- flag modules with no exercises
- flag concepts with weak evidence of distinctness
- suggest project concepts from capstone markers

View File

@ -1,65 +1,32 @@
# FAQ # FAQ
## What is Didactopus? ## Why add course ingestion?
Didactopus is a mastery-oriented learning infrastructure that uses concept graphs, evidence-based assessment, and adaptive planning to support serious learning. Because many open or user-supplied courses already encode:
- topic sequencing
- learning objectives
- exercises
- project prompts
- terminology
## Is this just a tutoring chatbot? That makes them strong starting material for draft domain packs.
No. The intended architecture is broader than tutoring. Didactopus maintains explicit representations of: ## Why not just embed all course text?
Because Didactopus needs structured artifacts:
- concepts - concepts
- prerequisites - prerequisites
- mastery criteria - projects
- evidence - rubrics
- learner state - mastery cues
- planning priorities
## How is an AI student's learned mastery represented? A flat embedding store is not enough for mastery planning.
An AI student's learned mastery is represented as structured state, not just conversation history. ## Why avoid PyKE or another heavy rule engine here?
Important elements include: Dependency stability matters. The current rule-policy adapter keeps rules simple,
- mastered concept set transparent, and dependency-light.
- evidence records
- dimension-level competence summaries
- weak-dimension lists
- project eligibility
- target-progress state
- produced artifacts and critiques
## Does Didactopus fine-tune the AI model? ## Can the rule layer be replaced later?
Not in the current design. Didactopus supervises and evaluates a learner agent, but it does not itself retrain foundation model weights. Yes. The adapter is designed so a future engine can be plugged in behind the same interface.
## Then how is the AI student “ready to work”?
Readiness is operationalized by the mastery state. An AI student is ready for a class of tasks when:
- relevant concepts are mastered
- confidence is high enough
- weak dimensions are acceptable for the target task
- prerequisite and project evidence support deployment
## Could mastered state be exported?
Yes. A future implementation should support export of:
- concept mastery ledgers
- evidence portfolios
- competence profiles
- project artifacts
- domain-specific capability summaries
## Is human learning treated the same way?
The same conceptual framework applies to both human and AI learners, though interfaces and evidence sources differ.
## What is the difference between mastery and model knowledge?
A model may contain latent knowledge or pattern familiarity. Didactopus mastery is narrower and stricter: it is evidence-backed demonstrated competence with respect to explicit concepts and criteria.
## Why not use only embeddings and LLM judgments?
Because correctness, especially in formal domains, often needs stronger guarantees than plausibility. That is why Didactopus may eventually need hybrid symbolic or executable validation components.
## Can Didactopus work offline?
Yes, that is a primary design goal. The architecture is local-first and can be paired with local model serving and locally stored domain packs.

31
docs/mastery-ledger.md Normal file
View File

@ -0,0 +1,31 @@
# Mastery Ledger
The mastery ledger is the structured record of what a learner has demonstrated.
## Core contents
- learner identity
- target domain or goal
- mastered concepts
- concept-level evidence summaries
- weak dimensions
- artifact records
- generated capability profile
## Exports
This scaffold exports:
- JSON capability profile
- Markdown capability report
- artifact manifest JSON
## Why it matters
The mastery ledger provides an explicit representation of readiness.
It supports both human and AI learners.
## Important caveat
The current scaffold is not a formal certification system. It is a structured
capability report driven by the Didactopus evidence and evaluator pipeline.

View File

@ -0,0 +1,35 @@
concepts:
- id: descriptive-statistics
title: Descriptive Statistics
description: Descriptive Statistics introduces measures of center and spread.
prerequisites: []
mastery_signals:
- Explain mean, median, and variance.
mastery_profile: {}
- id: probability-basics
title: Probability Basics
description: Probability Basics introduces events, likelihood, and Bayes-style reasoning.
prerequisites:
- descriptive-statistics
mastery_signals:
- Explain conditional probability.
mastery_profile: {}
- id: prior-and-posterior
title: Prior and Posterior
description: A Prior expresses assumptions before evidence. Posterior reasoning
updates belief after evidence.
prerequisites:
- probability-basics
mastery_signals:
- Explain a prior distribution.
- Explain how evidence changes belief.
mastery_profile: {}
- id: capstone-mini-project
title: Capstone Mini Project
description: This project asks learners to critique assumptions and produce a small
capstone artifact.
prerequisites:
- prior-and-posterior
mastery_signals:
- Write a short project report comparing priors and posteriors.
mastery_profile: {}

View File

@ -0,0 +1,5 @@
{
"source_name": "Sample Course",
"source_url": "",
"rights_note": "REVIEW REQUIRED"
}

View File

@ -0,0 +1,13 @@
name: introductory-bayesian-inference
display_name: Introductory Bayesian Inference
version: 0.1.0-draft
schema_version: '1'
didactopus_min_version: 0.1.0
didactopus_max_version: 0.9.99
description: Draft pack generated from sample course.
author: Wesley R. Elsberry
license: REVIEW-REQUIRED
dependencies: []
overrides: []
profile_templates: {}
cross_pack_links: []

View File

@ -0,0 +1,7 @@
projects:
- id: capstone-mini-project
title: Capstone Mini Project
difficulty: review-required
prerequisites: []
deliverables:
- project artifact

View File

@ -0,0 +1,3 @@
# Review Report
- Module 'Module 2: Bayesian Updating' appears to contain project-like material; review project extraction.

View File

@ -0,0 +1,17 @@
stages:
- id: stage-1
title: 'Module 1: Foundations'
concepts:
- descriptive-statistics
- probability-basics
checkpoint:
- Summarize a small dataset.
- Compute a simple conditional probability.
- id: stage-2
title: 'Module 2: Bayesian Updating'
concepts:
- prior-and-posterior
- capstone-mini-project
checkpoint:
- Compare prior and posterior beliefs.
- Write a short project report comparing priors and posteriors.

View File

@ -0,0 +1,6 @@
rubrics:
- id: draft-rubric
title: Draft Rubric
criteria:
- correctness
- explanation

23
examples/sample_course.md Normal file
View File

@ -0,0 +1,23 @@
# Introductory Bayesian Inference
## Module 1: Foundations
### Descriptive Statistics
- Objective: Explain mean, median, and variance.
- Exercise: Summarize a small dataset.
Descriptive Statistics introduces measures of center and spread.
### Probability Basics
- Objective: Explain conditional probability.
- Exercise: Compute a simple conditional probability.
Probability Basics introduces events, likelihood, and Bayes-style reasoning.
## Module 2: Bayesian Updating
### Prior and Posterior
- Objective: Explain a prior distribution.
- Objective: Explain how evidence changes belief.
- Exercise: Compare prior and posterior beliefs.
A Prior expresses assumptions before evidence. Posterior reasoning updates belief after evidence.
### Capstone Mini Project
- Exercise: Write a short project report comparing priors and posteriors.
This project asks learners to critique assumptions and produce a small capstone artifact.

View File

@ -5,21 +5,18 @@ build-backend = "setuptools.build_meta"
[project] [project]
name = "didactopus" name = "didactopus"
version = "0.1.0" version = "0.1.0"
description = "Didactopus: local-first AI-assisted autodidactic mastery platform" description = "Didactopus: course-to-pack ingestion scaffold"
readme = "README.md" readme = "README.md"
requires-python = ">=3.10" requires-python = ">=3.10"
license = {text = "MIT"} license = {text = "MIT"}
authors = [{name = "Wesley R. Elsberry"}] authors = [{name = "Wesley R. Elsberry"}]
dependencies = [ dependencies = ["pydantic>=2.7", "pyyaml>=6.0"]
"pydantic>=2.7",
"pyyaml>=6.0",
"networkx>=3.2",
]
[project.optional-dependencies] [project.optional-dependencies]
dev = ["pytest>=8.0", "ruff>=0.6"] dev = ["pytest>=8.0", "ruff>=0.6"]
[project.scripts] [project.scripts]
didactopus = "didactopus.main:main" didactopus-course-ingest = "didactopus.main:main"
[tool.setuptools.packages.find] [tool.setuptools.packages.find]
where = ["src"] where = ["src"]

View File

@ -1,92 +1,132 @@
from __future__ import annotations
from dataclasses import dataclass, field from dataclasses import dataclass, field
from .planner import rank_next_concepts, PlannerWeights from .evaluator_pipeline import (
from .evidence_engine import EvidenceState, ConceptEvidenceSummary LearnerAttempt,
RubricEvaluator,
CodeTestEvaluator,
SymbolicRuleEvaluator,
CritiqueEvaluator,
PortfolioEvaluator,
run_pipeline,
aggregate,
)
@dataclass
class ConceptEvidenceSummary:
concept_key: str
weak_dimensions: list[str] = field(default_factory=list)
mastered: bool = False
aggregated: dict = field(default_factory=dict)
evaluators: list[str] = field(default_factory=list)
@dataclass
class EvidenceState:
summary_by_concept: dict[str, ConceptEvidenceSummary] = field(default_factory=dict)
resurfaced_concepts: set[str] = field(default_factory=set)
@dataclass @dataclass
class AgenticStudentState: class AgenticStudentState:
learner_id: str = "demo-agent"
display_name: str = "Demo Agentic Student"
mastered_concepts: set[str] = field(default_factory=set) mastered_concepts: set[str] = field(default_factory=set)
evidence_state: EvidenceState = field(default_factory=EvidenceState) evidence_state: EvidenceState = field(default_factory=EvidenceState)
attempt_history: list[dict] = field(default_factory=list) attempt_history: list[dict] = field(default_factory=list)
artifacts: list[dict] = field(default_factory=list)
def synthetic_attempt_for_concept(concept: str) -> dict: def synthetic_attempt_for_concept(concept: str) -> LearnerAttempt:
if "descriptive-statistics" in concept: if "descriptive-statistics" in concept:
weak = [] return LearnerAttempt(
mastered = True concept=concept,
elif "probability-basics" in concept: artifact_type="explanation",
weak = ["transfer"] content="Mean and variance summarize a dataset because they describe center and spread.",
mastered = False metadata={"deliverable_count": 1, "artifact_name": "descriptive_statistics_note.md"},
elif "prior" in concept:
weak = ["explanation", "transfer"]
mastered = False
elif "posterior" in concept:
weak = ["critique", "transfer"]
mastered = False
elif "model-checking" in concept:
weak = ["critique"]
mastered = False
else:
weak = ["correctness"]
mastered = False
return {"concept": concept, "mastered": mastered, "weak_dimensions": weak}
def integrate_attempt(state: AgenticStudentState, attempt: dict) -> None:
concept = attempt["concept"]
summary = ConceptEvidenceSummary(
concept_key=concept,
weak_dimensions=list(attempt["weak_dimensions"]),
mastered=bool(attempt["mastered"]),
)
state.evidence_state.summary_by_concept[concept] = summary
if summary.mastered:
state.mastered_concepts.add(concept)
state.evidence_state.resurfaced_concepts.discard(concept)
else:
if concept in state.mastered_concepts:
state.mastered_concepts.remove(concept)
state.evidence_state.resurfaced_concepts.add(concept)
state.attempt_history.append(attempt)
def run_agentic_learning_loop(
graph,
project_catalog: list[dict],
target_concepts: list[str],
weights: PlannerWeights,
max_steps: int = 5,
) -> AgenticStudentState:
state = AgenticStudentState()
for _ in range(max_steps):
weak_dimensions_by_concept = {
key: summary.weak_dimensions
for key, summary in state.evidence_state.summary_by_concept.items()
}
fragile = set(state.evidence_state.resurfaced_concepts)
ranked = rank_next_concepts(
graph=graph,
mastered=state.mastered_concepts,
targets=target_concepts,
weak_dimensions_by_concept=weak_dimensions_by_concept,
fragile_concepts=fragile,
project_catalog=project_catalog,
weights=weights,
) )
if not ranked: if "probability-basics" in concept:
break return LearnerAttempt(
concept=concept,
artifact_type="explanation",
content="Conditional probability changes because context changes the sample space.",
metadata={"deliverable_count": 1, "artifact_name": "probability_basics_note.md"},
)
if "prior" in concept:
return LearnerAttempt(
concept=concept,
artifact_type="explanation",
content="A prior is an assumption before evidence, but one limitation is bias.",
metadata={"deliverable_count": 1, "artifact_name": "prior_reflection.md"},
)
if "posterior" in concept:
return LearnerAttempt(
concept=concept,
artifact_type="symbolic",
content="Therefore posterior = updated belief after evidence, but one assumption may be model fit.",
metadata={"deliverable_count": 1, "artifact_name": "posterior_symbolic_note.md"},
)
return LearnerAttempt(
concept=concept,
artifact_type="critique",
content="A weakness is hidden assumptions; a limitation is poor fit; uncertainty remains.",
metadata={"deliverable_count": 2, "artifact_name": "critique_report.md"},
)
chosen = ranked[0]["concept"]
attempt = synthetic_attempt_for_concept(chosen) def evaluator_set_for_attempt(attempt: LearnerAttempt):
evaluators = [RubricEvaluator(), CritiqueEvaluator()]
if attempt.artifact_type == "code":
evaluators.append(CodeTestEvaluator())
if attempt.artifact_type == "symbolic":
evaluators.append(SymbolicRuleEvaluator())
if attempt.artifact_type in {"project", "portfolio", "critique"}:
evaluators.append(PortfolioEvaluator())
return evaluators
def integrate_attempt(state: AgenticStudentState, attempt: LearnerAttempt) -> None:
results = run_pipeline(attempt, evaluator_set_for_attempt(attempt))
aggregated = aggregate(results)
weak = [dim for dim, score in aggregated.items() if score < 0.75]
mastered = len(aggregated) > 0 and all(score >= 0.75 for score in aggregated.values())
summary = ConceptEvidenceSummary(
concept_key=attempt.concept,
weak_dimensions=weak,
mastered=mastered,
aggregated=aggregated,
evaluators=[r.evaluator_name for r in results],
)
state.evidence_state.summary_by_concept[attempt.concept] = summary
if mastered:
state.mastered_concepts.add(attempt.concept)
state.evidence_state.resurfaced_concepts.discard(attempt.concept)
else:
if attempt.concept in state.mastered_concepts:
state.mastered_concepts.remove(attempt.concept)
state.evidence_state.resurfaced_concepts.add(attempt.concept)
state.attempt_history.append({
"concept": attempt.concept,
"artifact_type": attempt.artifact_type,
"aggregated": aggregated,
"weak_dimensions": weak,
"mastered": mastered,
"evaluators": [r.evaluator_name for r in results],
})
state.artifacts.append({
"concept": attempt.concept,
"artifact_type": attempt.artifact_type,
"artifact_name": attempt.metadata.get("artifact_name", f"{attempt.concept}.txt"),
})
def run_demo_agentic_loop(concepts: list[str]) -> AgenticStudentState:
state = AgenticStudentState()
for concept in concepts:
attempt = synthetic_attempt_for_concept(concept)
integrate_attempt(state, attempt) integrate_attempt(state, attempt)
if all(target in state.mastered_concepts for target in target_concepts):
break
return state return state

View File

@ -3,45 +3,23 @@ from pydantic import BaseModel, Field
import yaml import yaml
class PlatformConfig(BaseModel): class CourseIngestConfig(BaseModel):
default_dimension_thresholds: dict[str, float] = Field( default_pack_author: str = "Unknown"
default_factory=lambda: { default_license: str = "REVIEW-REQUIRED"
"correctness": 0.8, min_term_length: int = 4
"explanation": 0.75, max_terms_per_lesson: int = 8
"transfer": 0.7,
"project_execution": 0.75,
"critique": 0.7,
}
)
class PlannerConfig(BaseModel): class RulePolicyConfig(BaseModel):
readiness_bonus: float = 2.0 enable_prerequisite_order_rule: bool = True
target_distance_weight: float = 1.0 enable_duplicate_term_merge_rule: bool = True
weak_dimension_bonus: float = 1.2 enable_project_detection_rule: bool = True
fragile_review_bonus: float = 1.5 enable_review_flags: bool = True
project_unlock_bonus: float = 0.8
semantic_similarity_weight: float = 1.0
class EvidenceConfig(BaseModel):
resurfacing_threshold: float = 0.55
confidence_threshold: float = 0.8
evidence_weights: dict[str, float] = Field(
default_factory=lambda: {
"explanation": 1.0,
"problem": 1.5,
"project": 2.5,
"transfer": 2.0,
}
)
recent_evidence_multiplier: float = 1.35
class AppConfig(BaseModel): class AppConfig(BaseModel):
platform: PlatformConfig = Field(default_factory=PlatformConfig) course_ingest: CourseIngestConfig = Field(default_factory=CourseIngestConfig)
planner: PlannerConfig = Field(default_factory=PlannerConfig) rule_policy: RulePolicyConfig = Field(default_factory=RulePolicyConfig)
evidence: EvidenceConfig = Field(default_factory=EvidenceConfig)
def load_config(path: str | Path) -> AppConfig: def load_config(path: str | Path) -> AppConfig:

View File

@ -0,0 +1,128 @@
from __future__ import annotations
import re
from .course_schema import NormalizedCourse, Module, Lesson, ConceptCandidate
HEADING_RE = re.compile(r"^(#{1,3})\s+(.*)$")
BULLET_RE = re.compile(r"^\s*[-*+]\s+(.*)$")
def slugify(text: str) -> str:
cleaned = re.sub(r"[^a-zA-Z0-9]+", "-", text.strip().lower()).strip("-")
return cleaned or "untitled"
def extract_key_terms(text: str, min_term_length: int = 4, max_terms: int = 8) -> list[str]:
candidates = re.findall(r"\b[A-Z][A-Za-z0-9\-]{%d,}\b" % (min_term_length - 1), text)
seen = set()
ordered = []
for term in candidates:
if term not in seen:
seen.add(term)
ordered.append(term)
if len(ordered) >= max_terms:
break
return ordered
def parse_markdown_course(text: str, title: str, source_name: str = "", source_url: str = "", rights_note: str = "") -> NormalizedCourse:
lines = text.splitlines()
modules: list[Module] = []
current_module: Module | None = None
current_lesson: Lesson | None = None
body_buffer: list[str] = []
def flush_body():
nonlocal body_buffer, current_lesson
if current_lesson is not None and body_buffer:
current_lesson.body = "\n".join(body_buffer).strip()
body_buffer = []
for line in lines:
m = HEADING_RE.match(line)
if m:
level = len(m.group(1))
heading = m.group(2).strip()
if level == 1:
continue
elif level == 2:
flush_body()
if current_lesson is not None and current_module is not None:
current_module.lessons.append(current_lesson)
current_lesson = None
if current_module is not None:
modules.append(current_module)
current_module = Module(title=heading, lessons=[])
elif level == 3:
flush_body()
if current_lesson is not None and current_module is not None:
current_module.lessons.append(current_lesson)
current_lesson = Lesson(title=heading)
continue
bullet = BULLET_RE.match(line)
if bullet and current_lesson is not None:
item = bullet.group(1).strip()
lower = item.lower()
if lower.startswith("objective:"):
current_lesson.objectives.append(item.split(":", 1)[1].strip())
elif lower.startswith("exercise:"):
current_lesson.exercises.append(item.split(":", 1)[1].strip())
else:
body_buffer.append(line)
else:
body_buffer.append(line)
flush_body()
if current_lesson is not None and current_module is not None:
current_module.lessons.append(current_lesson)
if current_module is not None:
modules.append(current_module)
course = NormalizedCourse(
title=title,
source_name=source_name,
source_url=source_url,
rights_note=rights_note,
modules=modules,
)
for module in course.modules:
for lesson in module.lessons:
lesson.key_terms = extract_key_terms(f"{lesson.title}\n{lesson.body}")
return course
def extract_concept_candidates(course: NormalizedCourse) -> list[ConceptCandidate]:
concepts: list[ConceptCandidate] = []
seen_ids: set[str] = set()
for module in course.modules:
for lesson in module.lessons:
title_id = slugify(lesson.title)
if title_id not in seen_ids:
seen_ids.add(title_id)
concepts.append(
ConceptCandidate(
id=title_id,
title=lesson.title,
description=lesson.body[:240].strip(),
source_modules=[module.title],
source_lessons=[lesson.title],
mastery_signals=list(lesson.objectives[:3] or lesson.exercises[:2]),
)
)
for term in lesson.key_terms:
term_id = slugify(term)
if term_id in seen_ids:
continue
seen_ids.add(term_id)
concepts.append(
ConceptCandidate(
id=term_id,
title=term,
description=f"Candidate concept extracted from lesson '{lesson.title}'.",
source_modules=[module.title],
source_lessons=[lesson.title],
mastery_signals=list(lesson.objectives[:2]),
)
)
return concepts

View File

@ -0,0 +1,44 @@
from __future__ import annotations
from pydantic import BaseModel, Field
class Lesson(BaseModel):
title: str
body: str = ""
objectives: list[str] = Field(default_factory=list)
exercises: list[str] = Field(default_factory=list)
key_terms: list[str] = Field(default_factory=list)
class Module(BaseModel):
title: str
lessons: list[Lesson] = Field(default_factory=list)
class NormalizedCourse(BaseModel):
title: str
source_name: str = ""
source_url: str = ""
rights_note: str = ""
modules: list[Module] = Field(default_factory=list)
class ConceptCandidate(BaseModel):
id: str
title: str
description: str = ""
source_modules: list[str] = Field(default_factory=list)
source_lessons: list[str] = Field(default_factory=list)
prerequisites: list[str] = Field(default_factory=list)
mastery_signals: list[str] = Field(default_factory=list)
class DraftPack(BaseModel):
pack: dict
concepts: dict
roadmap: dict
projects: dict
rubrics: dict
review_report: list[str] = Field(default_factory=list)
attribution: dict = Field(default_factory=dict)

View File

@ -1,5 +1,6 @@
from dataclasses import dataclass, field from dataclasses import dataclass, field
@dataclass @dataclass
class LearnerAttempt: class LearnerAttempt:
concept: str concept: str
@ -7,6 +8,7 @@ class LearnerAttempt:
content: str content: str
metadata: dict = field(default_factory=dict) metadata: dict = field(default_factory=dict)
@dataclass @dataclass
class EvaluatorResult: class EvaluatorResult:
evaluator_name: str evaluator_name: str
@ -14,59 +16,84 @@ class EvaluatorResult:
passed: bool | None = None passed: bool | None = None
notes: str = "" notes: str = ""
class RubricEvaluator: class RubricEvaluator:
name = "rubric" name = "rubric"
def evaluate(self, attempt: LearnerAttempt): def evaluate(self, attempt: LearnerAttempt):
explanation = 0.85 if len(attempt.content) > 40 else 0.55 explanation = 0.85 if len(attempt.content.strip()) > 40 else 0.55
correctness = 0.80 if "because" in attempt.content.lower() else 0.65 correctness = 0.80 if "because" in attempt.content.lower() or "therefore" in attempt.content.lower() else 0.65
return EvaluatorResult(self.name, return EvaluatorResult(
{"correctness": correctness, self.name,
"explanation": explanation}) {"correctness": correctness, "explanation": explanation},
notes="Heuristic scaffold rubric score.",
)
class CodeTestEvaluator: class CodeTestEvaluator:
name = "code_test" name = "code_test"
def evaluate(self, attempt: LearnerAttempt): def evaluate(self, attempt: LearnerAttempt):
passed = "return" in attempt.content passed = "return" in attempt.content or "assert" in attempt.content
score = 0.9 if passed else 0.35 score = 0.9 if passed else 0.35
return EvaluatorResult(self.name, return EvaluatorResult(
{"correctness": score, self.name,
"project_execution": score}, {"correctness": score, "project_execution": score},
passed=passed) passed=passed,
notes="Stub code/test evaluator.",
)
class SymbolicRuleEvaluator: class SymbolicRuleEvaluator:
name = "symbolic_rule" name = "symbolic_rule"
def evaluate(self, attempt: LearnerAttempt): def evaluate(self, attempt: LearnerAttempt):
passed = "=" in attempt.content passed = "=" in attempt.content or "therefore" in attempt.content.lower()
score = 0.88 if passed else 0.4 score = 0.88 if passed else 0.4
return EvaluatorResult(self.name, return EvaluatorResult(
{"correctness": score}, self.name,
passed=passed) {"correctness": score},
passed=passed,
notes="Stub symbolic evaluator.",
)
class CritiqueEvaluator: class CritiqueEvaluator:
name = "critique" name = "critique"
def evaluate(self, attempt: LearnerAttempt): def evaluate(self, attempt: LearnerAttempt):
markers = ["assumption","bias","limitation","weakness"] markers = ["assumption", "bias", "limitation", "weakness", "uncertain"]
found = sum(m in attempt.content.lower() for m in markers) found = sum(m in attempt.content.lower() for m in markers)
score = min(1.0, 0.35 + 0.15 * found) score = min(1.0, 0.35 + 0.15 * found)
return EvaluatorResult(self.name, {"critique": score}) return EvaluatorResult(
self.name,
{"critique": score},
notes="Stub critique evaluator.",
)
class PortfolioEvaluator: class PortfolioEvaluator:
name = "portfolio" name = "portfolio"
def evaluate(self, attempt: LearnerAttempt): def evaluate(self, attempt: LearnerAttempt):
count = int(attempt.metadata.get("deliverable_count",1)) deliverable_count = int(attempt.metadata.get("deliverable_count", 1))
score = min(1.0, 0.5 + 0.1 * count) score = min(1.0, 0.5 + 0.1 * deliverable_count)
return EvaluatorResult(self.name, return EvaluatorResult(
{"project_execution": score, self.name,
"transfer": max(0.4, score-0.1)}) {"project_execution": score, "transfer": max(0.4, score - 0.1)},
notes="Stub portfolio evaluator.",
)
def run_pipeline(attempt, evaluators): def run_pipeline(attempt, evaluators):
return [e.evaluate(attempt) for e in evaluators] return [e.evaluate(attempt) for e in evaluators]
def aggregate(results): def aggregate(results):
totals = {} totals = {}
counts = {} counts = {}
for r in results: for r in results:
for d,v in r.dimensions.items(): for dim, val in r.dimensions.items():
totals[d] = totals.get(d,0)+v totals[dim] = totals.get(dim, 0.0) + val
counts[d] = counts.get(d,0)+1 counts[dim] = counts.get(dim, 0) + 1
return {d: totals[d]/counts[d] for d in totals} return {dim: totals[dim] / counts[dim] for dim in totals}

View File

@ -1,70 +1,65 @@
from __future__ import annotations
import argparse import argparse
import os
from pathlib import Path from pathlib import Path
from .agentic_loop import run_agentic_learning_loop
from .artifact_registry import check_pack_dependencies, detect_dependency_cycles, discover_domain_packs
from .config import load_config from .config import load_config
from .graph_builder import build_concept_graph from .course_ingest import parse_markdown_course, extract_concept_candidates
from .learning_graph import build_merged_learning_graph from .rule_policy import RuleContext, build_default_rules, run_rules
from .planner import PlannerWeights from .pack_emitter import build_draft_pack, write_draft_pack
def build_parser() -> argparse.ArgumentParser: def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(description="Didactopus agentic learner loop") parser = argparse.ArgumentParser(description="Didactopus course-to-pack ingestion pipeline")
parser.add_argument("--target", default="bayes-extension::posterior") parser.add_argument("--input", required=True)
parser.add_argument("--steps", type=int, default=5) parser.add_argument("--title", required=True)
parser.add_argument("--config", default=os.environ.get("DIDACTOPUS_CONFIG", "configs/config.example.yaml")) parser.add_argument("--source-name", default="")
parser.add_argument("--source-url", default="")
parser.add_argument("--rights-note", default="REVIEW REQUIRED")
parser.add_argument("--output-dir", default="generated-pack")
parser.add_argument("--config", default="configs/config.example.yaml")
return parser return parser
def main() -> None: def main() -> None:
args = build_parser().parse_args() args = build_parser().parse_args()
config = load_config(Path(args.config)) config = load_config(args.config)
results = discover_domain_packs(["domain-packs"]) text = Path(args.input).read_text(encoding="utf-8")
dep_errors = check_pack_dependencies(results)
cycles = detect_dependency_cycles(results)
if dep_errors: course = parse_markdown_course(
print("Dependency errors:") text=text,
for err in dep_errors: title=args.title,
print(f"- {err}") source_name=args.source_name,
if cycles: source_url=args.source_url,
print("Dependency cycles:") rights_note=args.rights_note,
for cycle in cycles:
print(f"- {' -> '.join(cycle)}")
return
merged = build_merged_learning_graph(results, config.platform.default_dimension_thresholds)
graph = build_concept_graph(results, config.platform.default_dimension_thresholds)
state = run_agentic_learning_loop(
graph=graph,
project_catalog=merged.project_catalog,
target_concepts=[args.target],
weights=PlannerWeights(
readiness_bonus=config.planner.readiness_bonus,
target_distance_weight=config.planner.target_distance_weight,
weak_dimension_bonus=config.planner.weak_dimension_bonus,
fragile_review_bonus=config.planner.fragile_review_bonus,
project_unlock_bonus=config.planner.project_unlock_bonus,
semantic_similarity_weight=config.planner.semantic_similarity_weight,
),
max_steps=args.steps,
) )
concepts = extract_concept_candidates(course)
context = RuleContext(course=course, concepts=concepts)
print("== Didactopus Agentic Learner Loop ==") rules = build_default_rules(
print(f"Target: {args.target}") enable_prereq=config.rule_policy.enable_prerequisite_order_rule,
print(f"Steps executed: {len(state.attempt_history)}") enable_merge=config.rule_policy.enable_duplicate_term_merge_rule,
print() enable_projects=config.rule_policy.enable_project_detection_rule,
print("Mastered concepts:") enable_review=config.rule_policy.enable_review_flags,
if state.mastered_concepts: )
for item in sorted(state.mastered_concepts): run_rules(context, rules)
print(f"- {item}")
else: draft = build_draft_pack(
print("- none") course=course,
print() concepts=context.concepts,
print("Attempt history:") author=config.course_ingest.default_pack_author,
for item in state.attempt_history: license_name=config.course_ingest.default_license,
weak = ", ".join(item["weak_dimensions"]) if item["weak_dimensions"] else "none" review_flags=context.review_flags,
print(f"- {item['concept']}: mastered={item['mastered']}, weak={weak}") )
write_draft_pack(draft, args.output_dir)
print("== Didactopus Course-to-Pack Ingest ==")
print(f"Course: {course.title}")
print(f"Modules: {len(course.modules)}")
print(f"Concept candidates: {len(context.concepts)}")
print(f"Review flags: {len(context.review_flags)}")
print(f"Output dir: {args.output_dir}")
if __name__ == "__main__":
main()

View File

@ -0,0 +1,78 @@
from dataclasses import dataclass, field, asdict
from pathlib import Path
import json
@dataclass
class CapabilityProfile:
learner_id: str
display_name: str
domain: str
mastered_concepts: list[str] = field(default_factory=list)
weak_dimensions_by_concept: dict[str, list[str]] = field(default_factory=dict)
evaluator_summary_by_concept: dict[str, dict] = field(default_factory=dict)
artifacts: list[dict] = field(default_factory=list)
def build_capability_profile(state, domain: str) -> CapabilityProfile:
weak = {}
summaries = {}
for concept, summary in state.evidence_state.summary_by_concept.items():
weak[concept] = list(summary.weak_dimensions)
summaries[concept] = dict(summary.aggregated)
return CapabilityProfile(
learner_id=state.learner_id,
display_name=state.display_name,
domain=domain,
mastered_concepts=sorted(state.mastered_concepts),
weak_dimensions_by_concept=weak,
evaluator_summary_by_concept=summaries,
artifacts=list(state.artifacts),
)
def export_capability_profile_json(profile: CapabilityProfile, path: str) -> None:
Path(path).write_text(json.dumps(asdict(profile), indent=2), encoding="utf-8")
def export_capability_report_markdown(profile: CapabilityProfile, path: str) -> None:
lines = [
f"# Capability Profile: {profile.display_name}",
"",
f"- Learner ID: `{profile.learner_id}`",
f"- Domain: `{profile.domain}`",
"",
"## Mastered Concepts",
]
if profile.mastered_concepts:
lines.extend([f"- {c}" for c in profile.mastered_concepts])
else:
lines.append("- none")
lines.extend(["", "## Concept Summaries"])
if profile.evaluator_summary_by_concept:
for concept, dims in sorted(profile.evaluator_summary_by_concept.items()):
lines.append(f"### {concept}")
if dims:
for dim, score in sorted(dims.items()):
lines.append(f"- {dim}: {score:.2f}")
weak = profile.weak_dimensions_by_concept.get(concept, [])
lines.append(f"- weak dimensions: {', '.join(weak) if weak else 'none'}")
lines.append("")
else:
lines.append("- none")
lines.extend(["## Artifacts"])
if profile.artifacts:
for art in profile.artifacts:
lines.append(f"- {art['artifact_name']} ({art['artifact_type']}) for {art['concept']}")
else:
lines.append("- none")
Path(path).write_text("\n".join(lines), encoding="utf-8")
def export_artifact_manifest(profile: CapabilityProfile, path: str) -> None:
manifest = {
"learner_id": profile.learner_id,
"domain": profile.domain,
"artifacts": profile.artifacts,
}
Path(path).write_text(json.dumps(manifest, indent=2), encoding="utf-8")

View File

@ -0,0 +1,78 @@
from __future__ import annotations
from pathlib import Path
import json
import yaml
from .course_schema import NormalizedCourse, ConceptCandidate, DraftPack
def build_draft_pack(course: NormalizedCourse, concepts: list[ConceptCandidate], author: str, license_name: str, review_flags: list[str]) -> DraftPack:
pack_name = course.title.lower().replace(" ", "-")
pack = {
"name": pack_name,
"display_name": course.title,
"version": "0.1.0-draft",
"schema_version": "1",
"didactopus_min_version": "0.1.0",
"didactopus_max_version": "0.9.99",
"description": f"Draft pack generated from course source '{course.source_name or course.title}'.",
"author": author,
"license": license_name,
"dependencies": [],
"overrides": [],
"profile_templates": {},
"cross_pack_links": [],
}
concepts_yaml = {
"concepts": [
{
"id": c.id,
"title": c.title,
"description": c.description,
"prerequisites": c.prerequisites,
"mastery_signals": c.mastery_signals,
"mastery_profile": {},
}
for c in concepts
]
}
roadmap = {
"stages": [
{
"id": f"stage-{i+1}",
"title": module.title,
"concepts": [c.id for c in concepts if module.title in c.source_modules and c.title in c.source_lessons],
"checkpoint": [ex for lesson in module.lessons for ex in lesson.exercises[:2]],
}
for i, module in enumerate(course.modules)
]
}
project_items = []
for module in course.modules:
for lesson in module.lessons:
text = f"{lesson.title}\n{lesson.body}".lower()
if "project" in text or "capstone" in text:
project_items.append({
"id": lesson.title.lower().replace(" ", "-"),
"title": lesson.title,
"difficulty": "review-required",
"prerequisites": [],
"deliverables": ["project artifact"],
})
projects = {"projects": project_items}
rubrics = {"rubrics": [{"id": "draft-rubric", "title": "Draft Rubric", "criteria": ["correctness", "explanation"]}]}
attribution = {"source_name": course.source_name, "source_url": course.source_url, "rights_note": course.rights_note}
return DraftPack(pack=pack, concepts=concepts_yaml, roadmap=roadmap, projects=projects, rubrics=rubrics, review_report=review_flags, attribution=attribution)
def write_draft_pack(pack: DraftPack, outdir: str | Path) -> None:
out = Path(outdir)
out.mkdir(parents=True, exist_ok=True)
(out / "pack.yaml").write_text(yaml.safe_dump(pack.pack, sort_keys=False), encoding="utf-8")
(out / "concepts.yaml").write_text(yaml.safe_dump(pack.concepts, sort_keys=False), encoding="utf-8")
(out / "roadmap.yaml").write_text(yaml.safe_dump(pack.roadmap, sort_keys=False), encoding="utf-8")
(out / "projects.yaml").write_text(yaml.safe_dump(pack.projects, sort_keys=False), encoding="utf-8")
(out / "rubrics.yaml").write_text(yaml.safe_dump(pack.rubrics, sort_keys=False), encoding="utf-8")
review_lines = ["# Review Report", ""] + [f"- {flag}" for flag in pack.review_report] if pack.review_report else ["# Review Report", "", "- none"]
(out / "review_report.md").write_text("\n".join(review_lines), encoding="utf-8")
(out / "license_attribution.json").write_text(json.dumps(pack.attribution, indent=2), encoding="utf-8")

View File

@ -0,0 +1,83 @@
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Callable
from .course_schema import NormalizedCourse, ConceptCandidate
@dataclass
class RuleContext:
course: NormalizedCourse
concepts: list[ConceptCandidate]
review_flags: list[str] = field(default_factory=list)
@dataclass
class Rule:
name: str
predicate: Callable[[RuleContext], bool]
action: Callable[[RuleContext], None]
def order_based_prerequisite_rule(context: RuleContext) -> None:
concept_titles = {c.title: c for c in context.concepts}
previous = None
for module in context.course.modules:
for lesson in module.lessons:
current = concept_titles.get(lesson.title)
if current is not None and previous is not None and previous.id not in current.prerequisites:
current.prerequisites.append(previous.id)
if current is not None:
previous = current
def duplicate_term_merge_rule(context: RuleContext) -> None:
seen = {}
deduped = []
for concept in context.concepts:
key = concept.title.strip().lower()
if key in seen:
seen[key].source_modules.extend(x for x in concept.source_modules if x not in seen[key].source_modules)
seen[key].source_lessons.extend(x for x in concept.source_lessons if x not in seen[key].source_lessons)
if concept.description and len(seen[key].description) < len(concept.description):
seen[key].description = concept.description
else:
seen[key] = concept
deduped.append(concept)
context.concepts[:] = deduped
def project_detection_rule(context: RuleContext) -> None:
for module in context.course.modules:
joined = " ".join(lesson.body for lesson in module.lessons).lower()
if "project" in joined or "capstone" in joined:
context.review_flags.append(f"Module '{module.title}' appears to contain project-like material; review project extraction.")
def review_flag_rule(context: RuleContext) -> None:
for module in context.course.modules:
if not any(lesson.exercises for lesson in module.lessons):
context.review_flags.append(f"Module '{module.title}' has no explicit exercises; mastery signals may be weak.")
for concept in context.concepts:
if not concept.mastery_signals:
context.review_flags.append(f"Concept '{concept.title}' has no extracted mastery signals; review manually.")
def build_default_rules(enable_prereq=True, enable_merge=True, enable_projects=True, enable_review=True) -> list[Rule]:
rules = []
if enable_prereq:
rules.append(Rule("order_based_prerequisite_rule", lambda ctx: True, order_based_prerequisite_rule))
if enable_merge:
rules.append(Rule("duplicate_term_merge_rule", lambda ctx: True, duplicate_term_merge_rule))
if enable_projects:
rules.append(Rule("project_detection_rule", lambda ctx: True, project_detection_rule))
if enable_review:
rules.append(Rule("review_flag_rule", lambda ctx: True, review_flag_rule))
return rules
def run_rules(context: RuleContext, rules: list[Rule]) -> RuleContext:
for rule in rules:
if rule.predicate(context):
rule.action(context)
return context

View File

@ -0,0 +1,26 @@
from didactopus.course_ingest import parse_markdown_course, extract_concept_candidates
SAMPLE = '''
# Sample Course
## Module 1
### Lesson A
- Objective: Explain Topic A.
- Exercise: Do task A.
Topic A body.
### Lesson B
- Objective: Explain Topic B.
Topic B body.
'''
def test_parse_markdown_course() -> None:
course = parse_markdown_course(SAMPLE, "Sample Course")
assert course.title == "Sample Course"
assert len(course.modules) == 1
assert len(course.modules[0].lessons) == 2
def test_extract_concepts() -> None:
course = parse_markdown_course(SAMPLE, "Sample Course")
concepts = extract_concept_candidates(course)
assert len(concepts) >= 2

View File

@ -0,0 +1,43 @@
from pathlib import Path
import json
from didactopus.agentic_loop import run_demo_agentic_loop
from didactopus.mastery_ledger import (
build_capability_profile,
export_capability_profile_json,
export_capability_report_markdown,
export_artifact_manifest,
)
def test_build_capability_profile() -> None:
state = run_demo_agentic_loop([
"foundations-statistics::descriptive-statistics",
"bayes-extension::prior",
])
profile = build_capability_profile(state, "Bayesian inference")
assert profile.domain == "Bayesian inference"
assert len(profile.artifacts) == 2
def test_exports(tmp_path: Path) -> None:
state = run_demo_agentic_loop([
"foundations-statistics::descriptive-statistics",
"bayes-extension::prior",
])
profile = build_capability_profile(state, "Bayesian inference")
json_path = tmp_path / "capability_profile.json"
md_path = tmp_path / "capability_report.md"
manifest_path = tmp_path / "artifact_manifest.json"
export_capability_profile_json(profile, str(json_path))
export_capability_report_markdown(profile, str(md_path))
export_artifact_manifest(profile, str(manifest_path))
assert json_path.exists()
assert md_path.exists()
assert manifest_path.exists()
data = json.loads(json_path.read_text(encoding="utf-8"))
assert data["domain"] == "Bayesian inference"

View File

@ -0,0 +1,24 @@
from pathlib import Path
from didactopus.course_ingest import parse_markdown_course, extract_concept_candidates
from didactopus.rule_policy import RuleContext, build_default_rules, run_rules
from didactopus.pack_emitter import build_draft_pack, write_draft_pack
SAMPLE = '''
# Sample Course
## Module 1
### Lesson A
- Objective: Explain Topic A.
- Exercise: Do task A.
Topic A body.
'''
def test_emit_pack(tmp_path: Path) -> None:
course = parse_markdown_course(SAMPLE, "Sample Course")
concepts = extract_concept_candidates(course)
ctx = RuleContext(course=course, concepts=concepts)
run_rules(ctx, build_default_rules())
draft = build_draft_pack(course, ctx.concepts, "Tester", "REVIEW", ctx.review_flags)
write_draft_pack(draft, tmp_path)
assert (tmp_path / "pack.yaml").exists()
assert (tmp_path / "review_report.md").exists()

24
tests/test_rule_policy.py Normal file
View File

@ -0,0 +1,24 @@
from didactopus.course_ingest import parse_markdown_course, extract_concept_candidates
from didactopus.rule_policy import RuleContext, build_default_rules, run_rules
SAMPLE = '''
# Sample Course
## Module 1
### Lesson A
- Objective: Explain Topic A.
- Exercise: Do task A.
Topic A body.
### Lesson B
- Objective: Explain Topic B.
- Exercise: Do task B.
Topic B body.
'''
def test_rules_run() -> None:
course = parse_markdown_course(SAMPLE, "Sample Course")
concepts = extract_concept_candidates(course)
ctx = RuleContext(course=course, concepts=concepts)
run_rules(ctx, build_default_rules())
assert len(ctx.concepts) >= 2