Compare commits
No commits in common. "8b4359f4cce1fa4a9d33994e21f586a8c9128958" and "687ed001fa58557337b94b0c0684b483c5dbc102" have entirely different histories.
8b4359f4cc
...
687ed001fa
94
README.md
94
README.md
|
|
@ -8,101 +8,10 @@
|
||||||
|
|
||||||
## Recent revisions
|
## Recent revisions
|
||||||
|
|
||||||
### Course Ingestion Pipeline
|
|
||||||
|
|
||||||
This revision adds a **Course-to-Pack Ingestion Pipeline** plus a **stable rule-policy adapter layer**.
|
|
||||||
|
|
||||||
The design goal is to turn open or user-supplied course materials into draft
|
|
||||||
Didactopus domain packs without introducing a brittle external rule-engine dependency.
|
|
||||||
|
|
||||||
#### Why no third-party rule engine here?
|
|
||||||
|
|
||||||
To minimize dependency risk, this scaffold uses a small declarative rule-policy
|
|
||||||
adapter implemented in pure Python and standard-library data structures.
|
|
||||||
|
|
||||||
That gives Didactopus:
|
|
||||||
- portable rules
|
|
||||||
- inspectable rule definitions
|
|
||||||
- deterministic behavior
|
|
||||||
- zero extra runtime dependency for policy evaluation
|
|
||||||
|
|
||||||
If a stronger rule engine is needed later, this adapter can remain the stable API surface.
|
|
||||||
|
|
||||||
#### What is included
|
|
||||||
|
|
||||||
- normalized course schema
|
|
||||||
- Markdown/HTML-ish text ingestion adapter
|
|
||||||
- module / lesson / objective extraction
|
|
||||||
- concept candidate extraction
|
|
||||||
- prerequisite guess generation
|
|
||||||
- rule-policy adapter
|
|
||||||
- draft pack emitter
|
|
||||||
- review report generation
|
|
||||||
- sample course input
|
|
||||||
- sample generated pack outputs
|
|
||||||
|
|
||||||
|
|
||||||
### Mastery Ledger
|
|
||||||
|
|
||||||
This revision adds a **Mastery Ledger + Capability Export** layer.
|
|
||||||
|
|
||||||
The main purpose is to let Didactopus turn accumulated learner state into
|
|
||||||
portable, inspectable artifacts that can support downstream deployment,
|
|
||||||
review, orchestration, or certification-like workflows.
|
|
||||||
|
|
||||||
#### What is new
|
|
||||||
|
|
||||||
- mastery ledger data model
|
|
||||||
- capability profile export
|
|
||||||
- JSON export of mastered concepts and evaluator summaries
|
|
||||||
- Markdown export of a readable capability report
|
|
||||||
- artifact manifest for produced deliverables
|
|
||||||
- demo CLI for generating exports for an AI student or human learner
|
|
||||||
- FAQ covering how learned mastery is represented and put to work
|
|
||||||
|
|
||||||
#### Why this matters
|
|
||||||
|
|
||||||
Didactopus can now do more than guide learning. It can also emit a structured
|
|
||||||
statement of what a learner appears able to do, based on explicit concepts,
|
|
||||||
evidence, and artifacts.
|
|
||||||
|
|
||||||
That makes it easier to use Didactopus as:
|
|
||||||
- a mastery tracker
|
|
||||||
- a portfolio generator
|
|
||||||
- a deployment-readiness aid
|
|
||||||
- an orchestration input for agent routing
|
|
||||||
|
|
||||||
#### Mastery representation
|
|
||||||
|
|
||||||
A learner's mastery is represented as structured operational state, including:
|
|
||||||
|
|
||||||
- mastered concepts
|
|
||||||
- evaluator results
|
|
||||||
- evidence summaries
|
|
||||||
- weak dimensions
|
|
||||||
- attempt history
|
|
||||||
- produced artifacts
|
|
||||||
- capability export
|
|
||||||
|
|
||||||
This is stricter than a normal chat transcript or self-description.
|
|
||||||
|
|
||||||
#### Future direction
|
|
||||||
|
|
||||||
A later revision should connect the capability export with:
|
|
||||||
- formal evaluator outputs
|
|
||||||
- signed evidence ledgers
|
|
||||||
- domain-specific capability schemas
|
|
||||||
- deployment policies for agent routing
|
|
||||||
|
|
||||||
|
|
||||||
### Evaluator Pipeline
|
|
||||||
|
|
||||||
This revision introduces a **pluggable evaluator pipeline** that converts
|
This revision introduces a **pluggable evaluator pipeline** that converts
|
||||||
learner attempts into structured mastery evidence.
|
learner attempts into structured mastery evidence.
|
||||||
|
|
||||||
### Agentic Learner Loop
|
The prior revision adds an **agentic learner loop** that turns Didactopus into a closed-loop mastery system prototype.
|
||||||
|
|
||||||
This revision adds an **agentic learner loop** that turns Didactopus into a closed-loop mastery system prototype.
|
|
||||||
|
|
||||||
The loop can now:
|
The loop can now:
|
||||||
|
|
||||||
|
|
@ -181,4 +90,3 @@ didactopus/
|
||||||
└── tests/
|
└── tests/
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,11 +1,18 @@
|
||||||
course_ingest:
|
model_provider:
|
||||||
default_pack_author: "Wesley R. Elsberry"
|
mode: local_first
|
||||||
default_license: "REVIEW-REQUIRED"
|
local:
|
||||||
min_term_length: 4
|
backend: ollama
|
||||||
max_terms_per_lesson: 8
|
endpoint: http://localhost:11434
|
||||||
|
model_name: llama3.1:8b
|
||||||
|
|
||||||
rule_policy:
|
platform:
|
||||||
enable_prerequisite_order_rule: true
|
default_dimension_thresholds:
|
||||||
enable_duplicate_term_merge_rule: true
|
correctness: 0.8
|
||||||
enable_project_detection_rule: true
|
explanation: 0.75
|
||||||
enable_review_flags: true
|
transfer: 0.7
|
||||||
|
project_execution: 0.75
|
||||||
|
critique: 0.7
|
||||||
|
|
||||||
|
artifacts:
|
||||||
|
local_pack_dirs:
|
||||||
|
- domain-packs
|
||||||
|
|
|
||||||
|
|
@ -1,35 +0,0 @@
|
||||||
# Course-to-Pack Ingestion Pipeline
|
|
||||||
|
|
||||||
The course-to-pack pipeline transforms educational material into Didactopus-native artifacts.
|
|
||||||
|
|
||||||
## Inputs
|
|
||||||
|
|
||||||
Typical sources:
|
|
||||||
- syllabus text
|
|
||||||
- lesson outlines
|
|
||||||
- markdown notes
|
|
||||||
- HTML course pages
|
|
||||||
- assignment sheets
|
|
||||||
- quiz prompts
|
|
||||||
- lecture transcripts
|
|
||||||
|
|
||||||
## Normalized intermediate structure
|
|
||||||
|
|
||||||
The pipeline builds a `NormalizedCourse` object containing:
|
|
||||||
- title
|
|
||||||
- source metadata
|
|
||||||
- modules
|
|
||||||
- lessons
|
|
||||||
- learning objectives
|
|
||||||
- exercises
|
|
||||||
- key terms
|
|
||||||
- project prompts
|
|
||||||
|
|
||||||
## Rule-policy adapter
|
|
||||||
|
|
||||||
The pipeline includes a small rule layer for stable policy transforms such as:
|
|
||||||
- suggest prerequisites from ordering
|
|
||||||
- merge repeated key-term candidates
|
|
||||||
- flag modules with no exercises
|
|
||||||
- flag concepts with weak evidence of distinctness
|
|
||||||
- suggest project concepts from capstone markers
|
|
||||||
73
docs/faq.md
73
docs/faq.md
|
|
@ -1,32 +1,65 @@
|
||||||
# FAQ
|
# FAQ
|
||||||
|
|
||||||
## Why add course ingestion?
|
## What is Didactopus?
|
||||||
|
|
||||||
Because many open or user-supplied courses already encode:
|
Didactopus is a mastery-oriented learning infrastructure that uses concept graphs, evidence-based assessment, and adaptive planning to support serious learning.
|
||||||
- topic sequencing
|
|
||||||
- learning objectives
|
|
||||||
- exercises
|
|
||||||
- project prompts
|
|
||||||
- terminology
|
|
||||||
|
|
||||||
That makes them strong starting material for draft domain packs.
|
## Is this just a tutoring chatbot?
|
||||||
|
|
||||||
## Why not just embed all course text?
|
No. The intended architecture is broader than tutoring. Didactopus maintains explicit representations of:
|
||||||
|
|
||||||
Because Didactopus needs structured artifacts:
|
|
||||||
- concepts
|
- concepts
|
||||||
- prerequisites
|
- prerequisites
|
||||||
- projects
|
- mastery criteria
|
||||||
- rubrics
|
- evidence
|
||||||
- mastery cues
|
- learner state
|
||||||
|
- planning priorities
|
||||||
|
|
||||||
A flat embedding store is not enough for mastery planning.
|
## How is an AI student's learned mastery represented?
|
||||||
|
|
||||||
## Why avoid PyKE or another heavy rule engine here?
|
An AI student's learned mastery is represented as structured state, not just conversation history.
|
||||||
|
|
||||||
Dependency stability matters. The current rule-policy adapter keeps rules simple,
|
Important elements include:
|
||||||
transparent, and dependency-light.
|
- mastered concept set
|
||||||
|
- evidence records
|
||||||
|
- dimension-level competence summaries
|
||||||
|
- weak-dimension lists
|
||||||
|
- project eligibility
|
||||||
|
- target-progress state
|
||||||
|
- produced artifacts and critiques
|
||||||
|
|
||||||
## Can the rule layer be replaced later?
|
## Does Didactopus fine-tune the AI model?
|
||||||
|
|
||||||
Yes. The adapter is designed so a future engine can be plugged in behind the same interface.
|
Not in the current design. Didactopus supervises and evaluates a learner agent, but it does not itself retrain foundation model weights.
|
||||||
|
|
||||||
|
## Then how is the AI student “ready to work”?
|
||||||
|
|
||||||
|
Readiness is operationalized by the mastery state. An AI student is ready for a class of tasks when:
|
||||||
|
- relevant concepts are mastered
|
||||||
|
- confidence is high enough
|
||||||
|
- weak dimensions are acceptable for the target task
|
||||||
|
- prerequisite and project evidence support deployment
|
||||||
|
|
||||||
|
## Could mastered state be exported?
|
||||||
|
|
||||||
|
Yes. A future implementation should support export of:
|
||||||
|
- concept mastery ledgers
|
||||||
|
- evidence portfolios
|
||||||
|
- competence profiles
|
||||||
|
- project artifacts
|
||||||
|
- domain-specific capability summaries
|
||||||
|
|
||||||
|
## Is human learning treated the same way?
|
||||||
|
|
||||||
|
The same conceptual framework applies to both human and AI learners, though interfaces and evidence sources differ.
|
||||||
|
|
||||||
|
## What is the difference between mastery and model knowledge?
|
||||||
|
|
||||||
|
A model may contain latent knowledge or pattern familiarity. Didactopus mastery is narrower and stricter: it is evidence-backed demonstrated competence with respect to explicit concepts and criteria.
|
||||||
|
|
||||||
|
## Why not use only embeddings and LLM judgments?
|
||||||
|
|
||||||
|
Because correctness, especially in formal domains, often needs stronger guarantees than plausibility. That is why Didactopus may eventually need hybrid symbolic or executable validation components.
|
||||||
|
|
||||||
|
## Can Didactopus work offline?
|
||||||
|
|
||||||
|
Yes, that is a primary design goal. The architecture is local-first and can be paired with local model serving and locally stored domain packs.
|
||||||
|
|
|
||||||
|
|
@ -1,31 +0,0 @@
|
||||||
# Mastery Ledger
|
|
||||||
|
|
||||||
The mastery ledger is the structured record of what a learner has demonstrated.
|
|
||||||
|
|
||||||
## Core contents
|
|
||||||
|
|
||||||
- learner identity
|
|
||||||
- target domain or goal
|
|
||||||
- mastered concepts
|
|
||||||
- concept-level evidence summaries
|
|
||||||
- weak dimensions
|
|
||||||
- artifact records
|
|
||||||
- generated capability profile
|
|
||||||
|
|
||||||
## Exports
|
|
||||||
|
|
||||||
This scaffold exports:
|
|
||||||
|
|
||||||
- JSON capability profile
|
|
||||||
- Markdown capability report
|
|
||||||
- artifact manifest JSON
|
|
||||||
|
|
||||||
## Why it matters
|
|
||||||
|
|
||||||
The mastery ledger provides an explicit representation of readiness.
|
|
||||||
It supports both human and AI learners.
|
|
||||||
|
|
||||||
## Important caveat
|
|
||||||
|
|
||||||
The current scaffold is not a formal certification system. It is a structured
|
|
||||||
capability report driven by the Didactopus evidence and evaluator pipeline.
|
|
||||||
|
|
@ -1,35 +0,0 @@
|
||||||
concepts:
|
|
||||||
- id: descriptive-statistics
|
|
||||||
title: Descriptive Statistics
|
|
||||||
description: Descriptive Statistics introduces measures of center and spread.
|
|
||||||
prerequisites: []
|
|
||||||
mastery_signals:
|
|
||||||
- Explain mean, median, and variance.
|
|
||||||
mastery_profile: {}
|
|
||||||
- id: probability-basics
|
|
||||||
title: Probability Basics
|
|
||||||
description: Probability Basics introduces events, likelihood, and Bayes-style reasoning.
|
|
||||||
prerequisites:
|
|
||||||
- descriptive-statistics
|
|
||||||
mastery_signals:
|
|
||||||
- Explain conditional probability.
|
|
||||||
mastery_profile: {}
|
|
||||||
- id: prior-and-posterior
|
|
||||||
title: Prior and Posterior
|
|
||||||
description: A Prior expresses assumptions before evidence. Posterior reasoning
|
|
||||||
updates belief after evidence.
|
|
||||||
prerequisites:
|
|
||||||
- probability-basics
|
|
||||||
mastery_signals:
|
|
||||||
- Explain a prior distribution.
|
|
||||||
- Explain how evidence changes belief.
|
|
||||||
mastery_profile: {}
|
|
||||||
- id: capstone-mini-project
|
|
||||||
title: Capstone Mini Project
|
|
||||||
description: This project asks learners to critique assumptions and produce a small
|
|
||||||
capstone artifact.
|
|
||||||
prerequisites:
|
|
||||||
- prior-and-posterior
|
|
||||||
mastery_signals:
|
|
||||||
- Write a short project report comparing priors and posteriors.
|
|
||||||
mastery_profile: {}
|
|
||||||
|
|
@ -1,5 +0,0 @@
|
||||||
{
|
|
||||||
"source_name": "Sample Course",
|
|
||||||
"source_url": "",
|
|
||||||
"rights_note": "REVIEW REQUIRED"
|
|
||||||
}
|
|
||||||
|
|
@ -1,13 +0,0 @@
|
||||||
name: introductory-bayesian-inference
|
|
||||||
display_name: Introductory Bayesian Inference
|
|
||||||
version: 0.1.0-draft
|
|
||||||
schema_version: '1'
|
|
||||||
didactopus_min_version: 0.1.0
|
|
||||||
didactopus_max_version: 0.9.99
|
|
||||||
description: Draft pack generated from sample course.
|
|
||||||
author: Wesley R. Elsberry
|
|
||||||
license: REVIEW-REQUIRED
|
|
||||||
dependencies: []
|
|
||||||
overrides: []
|
|
||||||
profile_templates: {}
|
|
||||||
cross_pack_links: []
|
|
||||||
|
|
@ -1,7 +0,0 @@
|
||||||
projects:
|
|
||||||
- id: capstone-mini-project
|
|
||||||
title: Capstone Mini Project
|
|
||||||
difficulty: review-required
|
|
||||||
prerequisites: []
|
|
||||||
deliverables:
|
|
||||||
- project artifact
|
|
||||||
|
|
@ -1,3 +0,0 @@
|
||||||
# Review Report
|
|
||||||
|
|
||||||
- Module 'Module 2: Bayesian Updating' appears to contain project-like material; review project extraction.
|
|
||||||
|
|
@ -1,17 +0,0 @@
|
||||||
stages:
|
|
||||||
- id: stage-1
|
|
||||||
title: 'Module 1: Foundations'
|
|
||||||
concepts:
|
|
||||||
- descriptive-statistics
|
|
||||||
- probability-basics
|
|
||||||
checkpoint:
|
|
||||||
- Summarize a small dataset.
|
|
||||||
- Compute a simple conditional probability.
|
|
||||||
- id: stage-2
|
|
||||||
title: 'Module 2: Bayesian Updating'
|
|
||||||
concepts:
|
|
||||||
- prior-and-posterior
|
|
||||||
- capstone-mini-project
|
|
||||||
checkpoint:
|
|
||||||
- Compare prior and posterior beliefs.
|
|
||||||
- Write a short project report comparing priors and posteriors.
|
|
||||||
|
|
@ -1,6 +0,0 @@
|
||||||
rubrics:
|
|
||||||
- id: draft-rubric
|
|
||||||
title: Draft Rubric
|
|
||||||
criteria:
|
|
||||||
- correctness
|
|
||||||
- explanation
|
|
||||||
|
|
@ -1,23 +0,0 @@
|
||||||
# Introductory Bayesian Inference
|
|
||||||
|
|
||||||
## Module 1: Foundations
|
|
||||||
### Descriptive Statistics
|
|
||||||
- Objective: Explain mean, median, and variance.
|
|
||||||
- Exercise: Summarize a small dataset.
|
|
||||||
Descriptive Statistics introduces measures of center and spread.
|
|
||||||
|
|
||||||
### Probability Basics
|
|
||||||
- Objective: Explain conditional probability.
|
|
||||||
- Exercise: Compute a simple conditional probability.
|
|
||||||
Probability Basics introduces events, likelihood, and Bayes-style reasoning.
|
|
||||||
|
|
||||||
## Module 2: Bayesian Updating
|
|
||||||
### Prior and Posterior
|
|
||||||
- Objective: Explain a prior distribution.
|
|
||||||
- Objective: Explain how evidence changes belief.
|
|
||||||
- Exercise: Compare prior and posterior beliefs.
|
|
||||||
A Prior expresses assumptions before evidence. Posterior reasoning updates belief after evidence.
|
|
||||||
|
|
||||||
### Capstone Mini Project
|
|
||||||
- Exercise: Write a short project report comparing priors and posteriors.
|
|
||||||
This project asks learners to critique assumptions and produce a small capstone artifact.
|
|
||||||
|
|
@ -5,18 +5,21 @@ build-backend = "setuptools.build_meta"
|
||||||
[project]
|
[project]
|
||||||
name = "didactopus"
|
name = "didactopus"
|
||||||
version = "0.1.0"
|
version = "0.1.0"
|
||||||
description = "Didactopus: course-to-pack ingestion scaffold"
|
description = "Didactopus: local-first AI-assisted autodidactic mastery platform"
|
||||||
readme = "README.md"
|
readme = "README.md"
|
||||||
requires-python = ">=3.10"
|
requires-python = ">=3.10"
|
||||||
license = {text = "MIT"}
|
license = {text = "MIT"}
|
||||||
authors = [{name = "Wesley R. Elsberry"}]
|
authors = [{name = "Wesley R. Elsberry"}]
|
||||||
dependencies = ["pydantic>=2.7", "pyyaml>=6.0"]
|
dependencies = [
|
||||||
|
"pydantic>=2.7",
|
||||||
|
"pyyaml>=6.0",
|
||||||
|
"networkx>=3.2",
|
||||||
|
]
|
||||||
[project.optional-dependencies]
|
[project.optional-dependencies]
|
||||||
dev = ["pytest>=8.0", "ruff>=0.6"]
|
dev = ["pytest>=8.0", "ruff>=0.6"]
|
||||||
|
|
||||||
[project.scripts]
|
[project.scripts]
|
||||||
didactopus-course-ingest = "didactopus.main:main"
|
didactopus = "didactopus.main:main"
|
||||||
|
|
||||||
[tool.setuptools.packages.find]
|
[tool.setuptools.packages.find]
|
||||||
where = ["src"]
|
where = ["src"]
|
||||||
|
|
|
||||||
|
|
@ -1,132 +1,92 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
|
|
||||||
from .evaluator_pipeline import (
|
from .planner import rank_next_concepts, PlannerWeights
|
||||||
LearnerAttempt,
|
from .evidence_engine import EvidenceState, ConceptEvidenceSummary
|
||||||
RubricEvaluator,
|
|
||||||
CodeTestEvaluator,
|
|
||||||
SymbolicRuleEvaluator,
|
|
||||||
CritiqueEvaluator,
|
|
||||||
PortfolioEvaluator,
|
|
||||||
run_pipeline,
|
|
||||||
aggregate,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class ConceptEvidenceSummary:
|
|
||||||
concept_key: str
|
|
||||||
weak_dimensions: list[str] = field(default_factory=list)
|
|
||||||
mastered: bool = False
|
|
||||||
aggregated: dict = field(default_factory=dict)
|
|
||||||
evaluators: list[str] = field(default_factory=list)
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class EvidenceState:
|
|
||||||
summary_by_concept: dict[str, ConceptEvidenceSummary] = field(default_factory=dict)
|
|
||||||
resurfaced_concepts: set[str] = field(default_factory=set)
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
class AgenticStudentState:
|
class AgenticStudentState:
|
||||||
learner_id: str = "demo-agent"
|
|
||||||
display_name: str = "Demo Agentic Student"
|
|
||||||
mastered_concepts: set[str] = field(default_factory=set)
|
mastered_concepts: set[str] = field(default_factory=set)
|
||||||
evidence_state: EvidenceState = field(default_factory=EvidenceState)
|
evidence_state: EvidenceState = field(default_factory=EvidenceState)
|
||||||
attempt_history: list[dict] = field(default_factory=list)
|
attempt_history: list[dict] = field(default_factory=list)
|
||||||
artifacts: list[dict] = field(default_factory=list)
|
|
||||||
|
|
||||||
|
|
||||||
def synthetic_attempt_for_concept(concept: str) -> LearnerAttempt:
|
def synthetic_attempt_for_concept(concept: str) -> dict:
|
||||||
if "descriptive-statistics" in concept:
|
if "descriptive-statistics" in concept:
|
||||||
return LearnerAttempt(
|
weak = []
|
||||||
concept=concept,
|
mastered = True
|
||||||
artifact_type="explanation",
|
elif "probability-basics" in concept:
|
||||||
content="Mean and variance summarize a dataset because they describe center and spread.",
|
weak = ["transfer"]
|
||||||
metadata={"deliverable_count": 1, "artifact_name": "descriptive_statistics_note.md"},
|
mastered = False
|
||||||
)
|
elif "prior" in concept:
|
||||||
if "probability-basics" in concept:
|
weak = ["explanation", "transfer"]
|
||||||
return LearnerAttempt(
|
mastered = False
|
||||||
concept=concept,
|
elif "posterior" in concept:
|
||||||
artifact_type="explanation",
|
weak = ["critique", "transfer"]
|
||||||
content="Conditional probability changes because context changes the sample space.",
|
mastered = False
|
||||||
metadata={"deliverable_count": 1, "artifact_name": "probability_basics_note.md"},
|
elif "model-checking" in concept:
|
||||||
)
|
weak = ["critique"]
|
||||||
if "prior" in concept:
|
mastered = False
|
||||||
return LearnerAttempt(
|
|
||||||
concept=concept,
|
|
||||||
artifact_type="explanation",
|
|
||||||
content="A prior is an assumption before evidence, but one limitation is bias.",
|
|
||||||
metadata={"deliverable_count": 1, "artifact_name": "prior_reflection.md"},
|
|
||||||
)
|
|
||||||
if "posterior" in concept:
|
|
||||||
return LearnerAttempt(
|
|
||||||
concept=concept,
|
|
||||||
artifact_type="symbolic",
|
|
||||||
content="Therefore posterior = updated belief after evidence, but one assumption may be model fit.",
|
|
||||||
metadata={"deliverable_count": 1, "artifact_name": "posterior_symbolic_note.md"},
|
|
||||||
)
|
|
||||||
return LearnerAttempt(
|
|
||||||
concept=concept,
|
|
||||||
artifact_type="critique",
|
|
||||||
content="A weakness is hidden assumptions; a limitation is poor fit; uncertainty remains.",
|
|
||||||
metadata={"deliverable_count": 2, "artifact_name": "critique_report.md"},
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def evaluator_set_for_attempt(attempt: LearnerAttempt):
|
|
||||||
evaluators = [RubricEvaluator(), CritiqueEvaluator()]
|
|
||||||
if attempt.artifact_type == "code":
|
|
||||||
evaluators.append(CodeTestEvaluator())
|
|
||||||
if attempt.artifact_type == "symbolic":
|
|
||||||
evaluators.append(SymbolicRuleEvaluator())
|
|
||||||
if attempt.artifact_type in {"project", "portfolio", "critique"}:
|
|
||||||
evaluators.append(PortfolioEvaluator())
|
|
||||||
return evaluators
|
|
||||||
|
|
||||||
|
|
||||||
def integrate_attempt(state: AgenticStudentState, attempt: LearnerAttempt) -> None:
|
|
||||||
results = run_pipeline(attempt, evaluator_set_for_attempt(attempt))
|
|
||||||
aggregated = aggregate(results)
|
|
||||||
weak = [dim for dim, score in aggregated.items() if score < 0.75]
|
|
||||||
mastered = len(aggregated) > 0 and all(score >= 0.75 for score in aggregated.values())
|
|
||||||
|
|
||||||
summary = ConceptEvidenceSummary(
|
|
||||||
concept_key=attempt.concept,
|
|
||||||
weak_dimensions=weak,
|
|
||||||
mastered=mastered,
|
|
||||||
aggregated=aggregated,
|
|
||||||
evaluators=[r.evaluator_name for r in results],
|
|
||||||
)
|
|
||||||
state.evidence_state.summary_by_concept[attempt.concept] = summary
|
|
||||||
|
|
||||||
if mastered:
|
|
||||||
state.mastered_concepts.add(attempt.concept)
|
|
||||||
state.evidence_state.resurfaced_concepts.discard(attempt.concept)
|
|
||||||
else:
|
else:
|
||||||
if attempt.concept in state.mastered_concepts:
|
weak = ["correctness"]
|
||||||
state.mastered_concepts.remove(attempt.concept)
|
mastered = False
|
||||||
state.evidence_state.resurfaced_concepts.add(attempt.concept)
|
|
||||||
|
|
||||||
state.attempt_history.append({
|
return {"concept": concept, "mastered": mastered, "weak_dimensions": weak}
|
||||||
"concept": attempt.concept,
|
|
||||||
"artifact_type": attempt.artifact_type,
|
|
||||||
"aggregated": aggregated,
|
|
||||||
"weak_dimensions": weak,
|
|
||||||
"mastered": mastered,
|
|
||||||
"evaluators": [r.evaluator_name for r in results],
|
|
||||||
})
|
|
||||||
|
|
||||||
state.artifacts.append({
|
|
||||||
"concept": attempt.concept,
|
|
||||||
"artifact_type": attempt.artifact_type,
|
|
||||||
"artifact_name": attempt.metadata.get("artifact_name", f"{attempt.concept}.txt"),
|
|
||||||
})
|
|
||||||
|
|
||||||
|
|
||||||
def run_demo_agentic_loop(concepts: list[str]) -> AgenticStudentState:
|
def integrate_attempt(state: AgenticStudentState, attempt: dict) -> None:
|
||||||
|
concept = attempt["concept"]
|
||||||
|
summary = ConceptEvidenceSummary(
|
||||||
|
concept_key=concept,
|
||||||
|
weak_dimensions=list(attempt["weak_dimensions"]),
|
||||||
|
mastered=bool(attempt["mastered"]),
|
||||||
|
)
|
||||||
|
state.evidence_state.summary_by_concept[concept] = summary
|
||||||
|
if summary.mastered:
|
||||||
|
state.mastered_concepts.add(concept)
|
||||||
|
state.evidence_state.resurfaced_concepts.discard(concept)
|
||||||
|
else:
|
||||||
|
if concept in state.mastered_concepts:
|
||||||
|
state.mastered_concepts.remove(concept)
|
||||||
|
state.evidence_state.resurfaced_concepts.add(concept)
|
||||||
|
state.attempt_history.append(attempt)
|
||||||
|
|
||||||
|
|
||||||
|
def run_agentic_learning_loop(
|
||||||
|
graph,
|
||||||
|
project_catalog: list[dict],
|
||||||
|
target_concepts: list[str],
|
||||||
|
weights: PlannerWeights,
|
||||||
|
max_steps: int = 5,
|
||||||
|
) -> AgenticStudentState:
|
||||||
state = AgenticStudentState()
|
state = AgenticStudentState()
|
||||||
for concept in concepts:
|
|
||||||
attempt = synthetic_attempt_for_concept(concept)
|
for _ in range(max_steps):
|
||||||
|
weak_dimensions_by_concept = {
|
||||||
|
key: summary.weak_dimensions
|
||||||
|
for key, summary in state.evidence_state.summary_by_concept.items()
|
||||||
|
}
|
||||||
|
fragile = set(state.evidence_state.resurfaced_concepts)
|
||||||
|
|
||||||
|
ranked = rank_next_concepts(
|
||||||
|
graph=graph,
|
||||||
|
mastered=state.mastered_concepts,
|
||||||
|
targets=target_concepts,
|
||||||
|
weak_dimensions_by_concept=weak_dimensions_by_concept,
|
||||||
|
fragile_concepts=fragile,
|
||||||
|
project_catalog=project_catalog,
|
||||||
|
weights=weights,
|
||||||
|
)
|
||||||
|
if not ranked:
|
||||||
|
break
|
||||||
|
|
||||||
|
chosen = ranked[0]["concept"]
|
||||||
|
attempt = synthetic_attempt_for_concept(chosen)
|
||||||
integrate_attempt(state, attempt)
|
integrate_attempt(state, attempt)
|
||||||
|
|
||||||
|
if all(target in state.mastered_concepts for target in target_concepts):
|
||||||
|
break
|
||||||
|
|
||||||
return state
|
return state
|
||||||
|
|
|
||||||
|
|
@ -3,23 +3,45 @@ from pydantic import BaseModel, Field
|
||||||
import yaml
|
import yaml
|
||||||
|
|
||||||
|
|
||||||
class CourseIngestConfig(BaseModel):
|
class PlatformConfig(BaseModel):
|
||||||
default_pack_author: str = "Unknown"
|
default_dimension_thresholds: dict[str, float] = Field(
|
||||||
default_license: str = "REVIEW-REQUIRED"
|
default_factory=lambda: {
|
||||||
min_term_length: int = 4
|
"correctness": 0.8,
|
||||||
max_terms_per_lesson: int = 8
|
"explanation": 0.75,
|
||||||
|
"transfer": 0.7,
|
||||||
|
"project_execution": 0.75,
|
||||||
|
"critique": 0.7,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class RulePolicyConfig(BaseModel):
|
class PlannerConfig(BaseModel):
|
||||||
enable_prerequisite_order_rule: bool = True
|
readiness_bonus: float = 2.0
|
||||||
enable_duplicate_term_merge_rule: bool = True
|
target_distance_weight: float = 1.0
|
||||||
enable_project_detection_rule: bool = True
|
weak_dimension_bonus: float = 1.2
|
||||||
enable_review_flags: bool = True
|
fragile_review_bonus: float = 1.5
|
||||||
|
project_unlock_bonus: float = 0.8
|
||||||
|
semantic_similarity_weight: float = 1.0
|
||||||
|
|
||||||
|
|
||||||
|
class EvidenceConfig(BaseModel):
|
||||||
|
resurfacing_threshold: float = 0.55
|
||||||
|
confidence_threshold: float = 0.8
|
||||||
|
evidence_weights: dict[str, float] = Field(
|
||||||
|
default_factory=lambda: {
|
||||||
|
"explanation": 1.0,
|
||||||
|
"problem": 1.5,
|
||||||
|
"project": 2.5,
|
||||||
|
"transfer": 2.0,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
recent_evidence_multiplier: float = 1.35
|
||||||
|
|
||||||
|
|
||||||
class AppConfig(BaseModel):
|
class AppConfig(BaseModel):
|
||||||
course_ingest: CourseIngestConfig = Field(default_factory=CourseIngestConfig)
|
platform: PlatformConfig = Field(default_factory=PlatformConfig)
|
||||||
rule_policy: RulePolicyConfig = Field(default_factory=RulePolicyConfig)
|
planner: PlannerConfig = Field(default_factory=PlannerConfig)
|
||||||
|
evidence: EvidenceConfig = Field(default_factory=EvidenceConfig)
|
||||||
|
|
||||||
|
|
||||||
def load_config(path: str | Path) -> AppConfig:
|
def load_config(path: str | Path) -> AppConfig:
|
||||||
|
|
|
||||||
|
|
@ -1,128 +0,0 @@
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import re
|
|
||||||
from .course_schema import NormalizedCourse, Module, Lesson, ConceptCandidate
|
|
||||||
|
|
||||||
HEADING_RE = re.compile(r"^(#{1,3})\s+(.*)$")
|
|
||||||
BULLET_RE = re.compile(r"^\s*[-*+]\s+(.*)$")
|
|
||||||
|
|
||||||
|
|
||||||
def slugify(text: str) -> str:
|
|
||||||
cleaned = re.sub(r"[^a-zA-Z0-9]+", "-", text.strip().lower()).strip("-")
|
|
||||||
return cleaned or "untitled"
|
|
||||||
|
|
||||||
|
|
||||||
def extract_key_terms(text: str, min_term_length: int = 4, max_terms: int = 8) -> list[str]:
|
|
||||||
candidates = re.findall(r"\b[A-Z][A-Za-z0-9\-]{%d,}\b" % (min_term_length - 1), text)
|
|
||||||
seen = set()
|
|
||||||
ordered = []
|
|
||||||
for term in candidates:
|
|
||||||
if term not in seen:
|
|
||||||
seen.add(term)
|
|
||||||
ordered.append(term)
|
|
||||||
if len(ordered) >= max_terms:
|
|
||||||
break
|
|
||||||
return ordered
|
|
||||||
|
|
||||||
|
|
||||||
def parse_markdown_course(text: str, title: str, source_name: str = "", source_url: str = "", rights_note: str = "") -> NormalizedCourse:
|
|
||||||
lines = text.splitlines()
|
|
||||||
modules: list[Module] = []
|
|
||||||
current_module: Module | None = None
|
|
||||||
current_lesson: Lesson | None = None
|
|
||||||
body_buffer: list[str] = []
|
|
||||||
|
|
||||||
def flush_body():
|
|
||||||
nonlocal body_buffer, current_lesson
|
|
||||||
if current_lesson is not None and body_buffer:
|
|
||||||
current_lesson.body = "\n".join(body_buffer).strip()
|
|
||||||
body_buffer = []
|
|
||||||
|
|
||||||
for line in lines:
|
|
||||||
m = HEADING_RE.match(line)
|
|
||||||
if m:
|
|
||||||
level = len(m.group(1))
|
|
||||||
heading = m.group(2).strip()
|
|
||||||
if level == 1:
|
|
||||||
continue
|
|
||||||
elif level == 2:
|
|
||||||
flush_body()
|
|
||||||
if current_lesson is not None and current_module is not None:
|
|
||||||
current_module.lessons.append(current_lesson)
|
|
||||||
current_lesson = None
|
|
||||||
if current_module is not None:
|
|
||||||
modules.append(current_module)
|
|
||||||
current_module = Module(title=heading, lessons=[])
|
|
||||||
elif level == 3:
|
|
||||||
flush_body()
|
|
||||||
if current_lesson is not None and current_module is not None:
|
|
||||||
current_module.lessons.append(current_lesson)
|
|
||||||
current_lesson = Lesson(title=heading)
|
|
||||||
continue
|
|
||||||
|
|
||||||
bullet = BULLET_RE.match(line)
|
|
||||||
if bullet and current_lesson is not None:
|
|
||||||
item = bullet.group(1).strip()
|
|
||||||
lower = item.lower()
|
|
||||||
if lower.startswith("objective:"):
|
|
||||||
current_lesson.objectives.append(item.split(":", 1)[1].strip())
|
|
||||||
elif lower.startswith("exercise:"):
|
|
||||||
current_lesson.exercises.append(item.split(":", 1)[1].strip())
|
|
||||||
else:
|
|
||||||
body_buffer.append(line)
|
|
||||||
else:
|
|
||||||
body_buffer.append(line)
|
|
||||||
|
|
||||||
flush_body()
|
|
||||||
if current_lesson is not None and current_module is not None:
|
|
||||||
current_module.lessons.append(current_lesson)
|
|
||||||
if current_module is not None:
|
|
||||||
modules.append(current_module)
|
|
||||||
|
|
||||||
course = NormalizedCourse(
|
|
||||||
title=title,
|
|
||||||
source_name=source_name,
|
|
||||||
source_url=source_url,
|
|
||||||
rights_note=rights_note,
|
|
||||||
modules=modules,
|
|
||||||
)
|
|
||||||
for module in course.modules:
|
|
||||||
for lesson in module.lessons:
|
|
||||||
lesson.key_terms = extract_key_terms(f"{lesson.title}\n{lesson.body}")
|
|
||||||
return course
|
|
||||||
|
|
||||||
|
|
||||||
def extract_concept_candidates(course: NormalizedCourse) -> list[ConceptCandidate]:
|
|
||||||
concepts: list[ConceptCandidate] = []
|
|
||||||
seen_ids: set[str] = set()
|
|
||||||
for module in course.modules:
|
|
||||||
for lesson in module.lessons:
|
|
||||||
title_id = slugify(lesson.title)
|
|
||||||
if title_id not in seen_ids:
|
|
||||||
seen_ids.add(title_id)
|
|
||||||
concepts.append(
|
|
||||||
ConceptCandidate(
|
|
||||||
id=title_id,
|
|
||||||
title=lesson.title,
|
|
||||||
description=lesson.body[:240].strip(),
|
|
||||||
source_modules=[module.title],
|
|
||||||
source_lessons=[lesson.title],
|
|
||||||
mastery_signals=list(lesson.objectives[:3] or lesson.exercises[:2]),
|
|
||||||
)
|
|
||||||
)
|
|
||||||
for term in lesson.key_terms:
|
|
||||||
term_id = slugify(term)
|
|
||||||
if term_id in seen_ids:
|
|
||||||
continue
|
|
||||||
seen_ids.add(term_id)
|
|
||||||
concepts.append(
|
|
||||||
ConceptCandidate(
|
|
||||||
id=term_id,
|
|
||||||
title=term,
|
|
||||||
description=f"Candidate concept extracted from lesson '{lesson.title}'.",
|
|
||||||
source_modules=[module.title],
|
|
||||||
source_lessons=[lesson.title],
|
|
||||||
mastery_signals=list(lesson.objectives[:2]),
|
|
||||||
)
|
|
||||||
)
|
|
||||||
return concepts
|
|
||||||
|
|
@ -1,44 +0,0 @@
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from pydantic import BaseModel, Field
|
|
||||||
|
|
||||||
|
|
||||||
class Lesson(BaseModel):
|
|
||||||
title: str
|
|
||||||
body: str = ""
|
|
||||||
objectives: list[str] = Field(default_factory=list)
|
|
||||||
exercises: list[str] = Field(default_factory=list)
|
|
||||||
key_terms: list[str] = Field(default_factory=list)
|
|
||||||
|
|
||||||
|
|
||||||
class Module(BaseModel):
|
|
||||||
title: str
|
|
||||||
lessons: list[Lesson] = Field(default_factory=list)
|
|
||||||
|
|
||||||
|
|
||||||
class NormalizedCourse(BaseModel):
|
|
||||||
title: str
|
|
||||||
source_name: str = ""
|
|
||||||
source_url: str = ""
|
|
||||||
rights_note: str = ""
|
|
||||||
modules: list[Module] = Field(default_factory=list)
|
|
||||||
|
|
||||||
|
|
||||||
class ConceptCandidate(BaseModel):
|
|
||||||
id: str
|
|
||||||
title: str
|
|
||||||
description: str = ""
|
|
||||||
source_modules: list[str] = Field(default_factory=list)
|
|
||||||
source_lessons: list[str] = Field(default_factory=list)
|
|
||||||
prerequisites: list[str] = Field(default_factory=list)
|
|
||||||
mastery_signals: list[str] = Field(default_factory=list)
|
|
||||||
|
|
||||||
|
|
||||||
class DraftPack(BaseModel):
|
|
||||||
pack: dict
|
|
||||||
concepts: dict
|
|
||||||
roadmap: dict
|
|
||||||
projects: dict
|
|
||||||
rubrics: dict
|
|
||||||
review_report: list[str] = Field(default_factory=list)
|
|
||||||
attribution: dict = Field(default_factory=dict)
|
|
||||||
|
|
@ -1,6 +1,5 @@
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
class LearnerAttempt:
|
class LearnerAttempt:
|
||||||
concept: str
|
concept: str
|
||||||
|
|
@ -8,7 +7,6 @@ class LearnerAttempt:
|
||||||
content: str
|
content: str
|
||||||
metadata: dict = field(default_factory=dict)
|
metadata: dict = field(default_factory=dict)
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
class EvaluatorResult:
|
class EvaluatorResult:
|
||||||
evaluator_name: str
|
evaluator_name: str
|
||||||
|
|
@ -16,84 +14,59 @@ class EvaluatorResult:
|
||||||
passed: bool | None = None
|
passed: bool | None = None
|
||||||
notes: str = ""
|
notes: str = ""
|
||||||
|
|
||||||
|
|
||||||
class RubricEvaluator:
|
class RubricEvaluator:
|
||||||
name = "rubric"
|
name = "rubric"
|
||||||
|
|
||||||
def evaluate(self, attempt: LearnerAttempt):
|
def evaluate(self, attempt: LearnerAttempt):
|
||||||
explanation = 0.85 if len(attempt.content.strip()) > 40 else 0.55
|
explanation = 0.85 if len(attempt.content) > 40 else 0.55
|
||||||
correctness = 0.80 if "because" in attempt.content.lower() or "therefore" in attempt.content.lower() else 0.65
|
correctness = 0.80 if "because" in attempt.content.lower() else 0.65
|
||||||
return EvaluatorResult(
|
return EvaluatorResult(self.name,
|
||||||
self.name,
|
{"correctness": correctness,
|
||||||
{"correctness": correctness, "explanation": explanation},
|
"explanation": explanation})
|
||||||
notes="Heuristic scaffold rubric score.",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class CodeTestEvaluator:
|
class CodeTestEvaluator:
|
||||||
name = "code_test"
|
name = "code_test"
|
||||||
|
|
||||||
def evaluate(self, attempt: LearnerAttempt):
|
def evaluate(self, attempt: LearnerAttempt):
|
||||||
passed = "return" in attempt.content or "assert" in attempt.content
|
passed = "return" in attempt.content
|
||||||
score = 0.9 if passed else 0.35
|
score = 0.9 if passed else 0.35
|
||||||
return EvaluatorResult(
|
return EvaluatorResult(self.name,
|
||||||
self.name,
|
{"correctness": score,
|
||||||
{"correctness": score, "project_execution": score},
|
"project_execution": score},
|
||||||
passed=passed,
|
passed=passed)
|
||||||
notes="Stub code/test evaluator.",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class SymbolicRuleEvaluator:
|
class SymbolicRuleEvaluator:
|
||||||
name = "symbolic_rule"
|
name = "symbolic_rule"
|
||||||
|
|
||||||
def evaluate(self, attempt: LearnerAttempt):
|
def evaluate(self, attempt: LearnerAttempt):
|
||||||
passed = "=" in attempt.content or "therefore" in attempt.content.lower()
|
passed = "=" in attempt.content
|
||||||
score = 0.88 if passed else 0.4
|
score = 0.88 if passed else 0.4
|
||||||
return EvaluatorResult(
|
return EvaluatorResult(self.name,
|
||||||
self.name,
|
{"correctness": score},
|
||||||
{"correctness": score},
|
passed=passed)
|
||||||
passed=passed,
|
|
||||||
notes="Stub symbolic evaluator.",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class CritiqueEvaluator:
|
class CritiqueEvaluator:
|
||||||
name = "critique"
|
name = "critique"
|
||||||
|
|
||||||
def evaluate(self, attempt: LearnerAttempt):
|
def evaluate(self, attempt: LearnerAttempt):
|
||||||
markers = ["assumption", "bias", "limitation", "weakness", "uncertain"]
|
markers = ["assumption","bias","limitation","weakness"]
|
||||||
found = sum(m in attempt.content.lower() for m in markers)
|
found = sum(m in attempt.content.lower() for m in markers)
|
||||||
score = min(1.0, 0.35 + 0.15 * found)
|
score = min(1.0, 0.35 + 0.15 * found)
|
||||||
return EvaluatorResult(
|
return EvaluatorResult(self.name, {"critique": score})
|
||||||
self.name,
|
|
||||||
{"critique": score},
|
|
||||||
notes="Stub critique evaluator.",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class PortfolioEvaluator:
|
class PortfolioEvaluator:
|
||||||
name = "portfolio"
|
name = "portfolio"
|
||||||
|
|
||||||
def evaluate(self, attempt: LearnerAttempt):
|
def evaluate(self, attempt: LearnerAttempt):
|
||||||
deliverable_count = int(attempt.metadata.get("deliverable_count", 1))
|
count = int(attempt.metadata.get("deliverable_count",1))
|
||||||
score = min(1.0, 0.5 + 0.1 * deliverable_count)
|
score = min(1.0, 0.5 + 0.1 * count)
|
||||||
return EvaluatorResult(
|
return EvaluatorResult(self.name,
|
||||||
self.name,
|
{"project_execution": score,
|
||||||
{"project_execution": score, "transfer": max(0.4, score - 0.1)},
|
"transfer": max(0.4, score-0.1)})
|
||||||
notes="Stub portfolio evaluator.",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def run_pipeline(attempt, evaluators):
|
def run_pipeline(attempt, evaluators):
|
||||||
return [e.evaluate(attempt) for e in evaluators]
|
return [e.evaluate(attempt) for e in evaluators]
|
||||||
|
|
||||||
|
|
||||||
def aggregate(results):
|
def aggregate(results):
|
||||||
totals = {}
|
totals = {}
|
||||||
counts = {}
|
counts = {}
|
||||||
for r in results:
|
for r in results:
|
||||||
for dim, val in r.dimensions.items():
|
for d,v in r.dimensions.items():
|
||||||
totals[dim] = totals.get(dim, 0.0) + val
|
totals[d] = totals.get(d,0)+v
|
||||||
counts[dim] = counts.get(dim, 0) + 1
|
counts[d] = counts.get(d,0)+1
|
||||||
return {dim: totals[dim] / counts[dim] for dim in totals}
|
return {d: totals[d]/counts[d] for d in totals}
|
||||||
|
|
|
||||||
|
|
@ -1,65 +1,70 @@
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import argparse
|
import argparse
|
||||||
|
import os
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
|
from .agentic_loop import run_agentic_learning_loop
|
||||||
|
from .artifact_registry import check_pack_dependencies, detect_dependency_cycles, discover_domain_packs
|
||||||
from .config import load_config
|
from .config import load_config
|
||||||
from .course_ingest import parse_markdown_course, extract_concept_candidates
|
from .graph_builder import build_concept_graph
|
||||||
from .rule_policy import RuleContext, build_default_rules, run_rules
|
from .learning_graph import build_merged_learning_graph
|
||||||
from .pack_emitter import build_draft_pack, write_draft_pack
|
from .planner import PlannerWeights
|
||||||
|
|
||||||
|
|
||||||
def build_parser() -> argparse.ArgumentParser:
|
def build_parser() -> argparse.ArgumentParser:
|
||||||
parser = argparse.ArgumentParser(description="Didactopus course-to-pack ingestion pipeline")
|
parser = argparse.ArgumentParser(description="Didactopus agentic learner loop")
|
||||||
parser.add_argument("--input", required=True)
|
parser.add_argument("--target", default="bayes-extension::posterior")
|
||||||
parser.add_argument("--title", required=True)
|
parser.add_argument("--steps", type=int, default=5)
|
||||||
parser.add_argument("--source-name", default="")
|
parser.add_argument("--config", default=os.environ.get("DIDACTOPUS_CONFIG", "configs/config.example.yaml"))
|
||||||
parser.add_argument("--source-url", default="")
|
|
||||||
parser.add_argument("--rights-note", default="REVIEW REQUIRED")
|
|
||||||
parser.add_argument("--output-dir", default="generated-pack")
|
|
||||||
parser.add_argument("--config", default="configs/config.example.yaml")
|
|
||||||
return parser
|
return parser
|
||||||
|
|
||||||
|
|
||||||
def main() -> None:
|
def main() -> None:
|
||||||
args = build_parser().parse_args()
|
args = build_parser().parse_args()
|
||||||
config = load_config(args.config)
|
config = load_config(Path(args.config))
|
||||||
text = Path(args.input).read_text(encoding="utf-8")
|
results = discover_domain_packs(["domain-packs"])
|
||||||
|
dep_errors = check_pack_dependencies(results)
|
||||||
|
cycles = detect_dependency_cycles(results)
|
||||||
|
|
||||||
course = parse_markdown_course(
|
if dep_errors:
|
||||||
text=text,
|
print("Dependency errors:")
|
||||||
title=args.title,
|
for err in dep_errors:
|
||||||
source_name=args.source_name,
|
print(f"- {err}")
|
||||||
source_url=args.source_url,
|
if cycles:
|
||||||
rights_note=args.rights_note,
|
print("Dependency cycles:")
|
||||||
|
for cycle in cycles:
|
||||||
|
print(f"- {' -> '.join(cycle)}")
|
||||||
|
return
|
||||||
|
|
||||||
|
merged = build_merged_learning_graph(results, config.platform.default_dimension_thresholds)
|
||||||
|
graph = build_concept_graph(results, config.platform.default_dimension_thresholds)
|
||||||
|
|
||||||
|
state = run_agentic_learning_loop(
|
||||||
|
graph=graph,
|
||||||
|
project_catalog=merged.project_catalog,
|
||||||
|
target_concepts=[args.target],
|
||||||
|
weights=PlannerWeights(
|
||||||
|
readiness_bonus=config.planner.readiness_bonus,
|
||||||
|
target_distance_weight=config.planner.target_distance_weight,
|
||||||
|
weak_dimension_bonus=config.planner.weak_dimension_bonus,
|
||||||
|
fragile_review_bonus=config.planner.fragile_review_bonus,
|
||||||
|
project_unlock_bonus=config.planner.project_unlock_bonus,
|
||||||
|
semantic_similarity_weight=config.planner.semantic_similarity_weight,
|
||||||
|
),
|
||||||
|
max_steps=args.steps,
|
||||||
)
|
)
|
||||||
concepts = extract_concept_candidates(course)
|
|
||||||
context = RuleContext(course=course, concepts=concepts)
|
|
||||||
|
|
||||||
rules = build_default_rules(
|
print("== Didactopus Agentic Learner Loop ==")
|
||||||
enable_prereq=config.rule_policy.enable_prerequisite_order_rule,
|
print(f"Target: {args.target}")
|
||||||
enable_merge=config.rule_policy.enable_duplicate_term_merge_rule,
|
print(f"Steps executed: {len(state.attempt_history)}")
|
||||||
enable_projects=config.rule_policy.enable_project_detection_rule,
|
print()
|
||||||
enable_review=config.rule_policy.enable_review_flags,
|
print("Mastered concepts:")
|
||||||
)
|
if state.mastered_concepts:
|
||||||
run_rules(context, rules)
|
for item in sorted(state.mastered_concepts):
|
||||||
|
print(f"- {item}")
|
||||||
draft = build_draft_pack(
|
else:
|
||||||
course=course,
|
print("- none")
|
||||||
concepts=context.concepts,
|
print()
|
||||||
author=config.course_ingest.default_pack_author,
|
print("Attempt history:")
|
||||||
license_name=config.course_ingest.default_license,
|
for item in state.attempt_history:
|
||||||
review_flags=context.review_flags,
|
weak = ", ".join(item["weak_dimensions"]) if item["weak_dimensions"] else "none"
|
||||||
)
|
print(f"- {item['concept']}: mastered={item['mastered']}, weak={weak}")
|
||||||
write_draft_pack(draft, args.output_dir)
|
|
||||||
|
|
||||||
print("== Didactopus Course-to-Pack Ingest ==")
|
|
||||||
print(f"Course: {course.title}")
|
|
||||||
print(f"Modules: {len(course.modules)}")
|
|
||||||
print(f"Concept candidates: {len(context.concepts)}")
|
|
||||||
print(f"Review flags: {len(context.review_flags)}")
|
|
||||||
print(f"Output dir: {args.output_dir}")
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
main()
|
|
||||||
|
|
|
||||||
|
|
@ -1,78 +0,0 @@
|
||||||
from dataclasses import dataclass, field, asdict
|
|
||||||
from pathlib import Path
|
|
||||||
import json
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class CapabilityProfile:
|
|
||||||
learner_id: str
|
|
||||||
display_name: str
|
|
||||||
domain: str
|
|
||||||
mastered_concepts: list[str] = field(default_factory=list)
|
|
||||||
weak_dimensions_by_concept: dict[str, list[str]] = field(default_factory=dict)
|
|
||||||
evaluator_summary_by_concept: dict[str, dict] = field(default_factory=dict)
|
|
||||||
artifacts: list[dict] = field(default_factory=list)
|
|
||||||
|
|
||||||
|
|
||||||
def build_capability_profile(state, domain: str) -> CapabilityProfile:
|
|
||||||
weak = {}
|
|
||||||
summaries = {}
|
|
||||||
for concept, summary in state.evidence_state.summary_by_concept.items():
|
|
||||||
weak[concept] = list(summary.weak_dimensions)
|
|
||||||
summaries[concept] = dict(summary.aggregated)
|
|
||||||
return CapabilityProfile(
|
|
||||||
learner_id=state.learner_id,
|
|
||||||
display_name=state.display_name,
|
|
||||||
domain=domain,
|
|
||||||
mastered_concepts=sorted(state.mastered_concepts),
|
|
||||||
weak_dimensions_by_concept=weak,
|
|
||||||
evaluator_summary_by_concept=summaries,
|
|
||||||
artifacts=list(state.artifacts),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def export_capability_profile_json(profile: CapabilityProfile, path: str) -> None:
|
|
||||||
Path(path).write_text(json.dumps(asdict(profile), indent=2), encoding="utf-8")
|
|
||||||
|
|
||||||
|
|
||||||
def export_capability_report_markdown(profile: CapabilityProfile, path: str) -> None:
|
|
||||||
lines = [
|
|
||||||
f"# Capability Profile: {profile.display_name}",
|
|
||||||
"",
|
|
||||||
f"- Learner ID: `{profile.learner_id}`",
|
|
||||||
f"- Domain: `{profile.domain}`",
|
|
||||||
"",
|
|
||||||
"## Mastered Concepts",
|
|
||||||
]
|
|
||||||
if profile.mastered_concepts:
|
|
||||||
lines.extend([f"- {c}" for c in profile.mastered_concepts])
|
|
||||||
else:
|
|
||||||
lines.append("- none")
|
|
||||||
lines.extend(["", "## Concept Summaries"])
|
|
||||||
if profile.evaluator_summary_by_concept:
|
|
||||||
for concept, dims in sorted(profile.evaluator_summary_by_concept.items()):
|
|
||||||
lines.append(f"### {concept}")
|
|
||||||
if dims:
|
|
||||||
for dim, score in sorted(dims.items()):
|
|
||||||
lines.append(f"- {dim}: {score:.2f}")
|
|
||||||
weak = profile.weak_dimensions_by_concept.get(concept, [])
|
|
||||||
lines.append(f"- weak dimensions: {', '.join(weak) if weak else 'none'}")
|
|
||||||
lines.append("")
|
|
||||||
else:
|
|
||||||
lines.append("- none")
|
|
||||||
lines.extend(["## Artifacts"])
|
|
||||||
if profile.artifacts:
|
|
||||||
for art in profile.artifacts:
|
|
||||||
lines.append(f"- {art['artifact_name']} ({art['artifact_type']}) for {art['concept']}")
|
|
||||||
else:
|
|
||||||
lines.append("- none")
|
|
||||||
Path(path).write_text("\n".join(lines), encoding="utf-8")
|
|
||||||
|
|
||||||
|
|
||||||
def export_artifact_manifest(profile: CapabilityProfile, path: str) -> None:
|
|
||||||
manifest = {
|
|
||||||
"learner_id": profile.learner_id,
|
|
||||||
"domain": profile.domain,
|
|
||||||
"artifacts": profile.artifacts,
|
|
||||||
}
|
|
||||||
Path(path).write_text(json.dumps(manifest, indent=2), encoding="utf-8")
|
|
||||||
|
|
@ -1,78 +0,0 @@
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from pathlib import Path
|
|
||||||
import json
|
|
||||||
import yaml
|
|
||||||
from .course_schema import NormalizedCourse, ConceptCandidate, DraftPack
|
|
||||||
|
|
||||||
|
|
||||||
def build_draft_pack(course: NormalizedCourse, concepts: list[ConceptCandidate], author: str, license_name: str, review_flags: list[str]) -> DraftPack:
|
|
||||||
pack_name = course.title.lower().replace(" ", "-")
|
|
||||||
pack = {
|
|
||||||
"name": pack_name,
|
|
||||||
"display_name": course.title,
|
|
||||||
"version": "0.1.0-draft",
|
|
||||||
"schema_version": "1",
|
|
||||||
"didactopus_min_version": "0.1.0",
|
|
||||||
"didactopus_max_version": "0.9.99",
|
|
||||||
"description": f"Draft pack generated from course source '{course.source_name or course.title}'.",
|
|
||||||
"author": author,
|
|
||||||
"license": license_name,
|
|
||||||
"dependencies": [],
|
|
||||||
"overrides": [],
|
|
||||||
"profile_templates": {},
|
|
||||||
"cross_pack_links": [],
|
|
||||||
}
|
|
||||||
concepts_yaml = {
|
|
||||||
"concepts": [
|
|
||||||
{
|
|
||||||
"id": c.id,
|
|
||||||
"title": c.title,
|
|
||||||
"description": c.description,
|
|
||||||
"prerequisites": c.prerequisites,
|
|
||||||
"mastery_signals": c.mastery_signals,
|
|
||||||
"mastery_profile": {},
|
|
||||||
}
|
|
||||||
for c in concepts
|
|
||||||
]
|
|
||||||
}
|
|
||||||
roadmap = {
|
|
||||||
"stages": [
|
|
||||||
{
|
|
||||||
"id": f"stage-{i+1}",
|
|
||||||
"title": module.title,
|
|
||||||
"concepts": [c.id for c in concepts if module.title in c.source_modules and c.title in c.source_lessons],
|
|
||||||
"checkpoint": [ex for lesson in module.lessons for ex in lesson.exercises[:2]],
|
|
||||||
}
|
|
||||||
for i, module in enumerate(course.modules)
|
|
||||||
]
|
|
||||||
}
|
|
||||||
project_items = []
|
|
||||||
for module in course.modules:
|
|
||||||
for lesson in module.lessons:
|
|
||||||
text = f"{lesson.title}\n{lesson.body}".lower()
|
|
||||||
if "project" in text or "capstone" in text:
|
|
||||||
project_items.append({
|
|
||||||
"id": lesson.title.lower().replace(" ", "-"),
|
|
||||||
"title": lesson.title,
|
|
||||||
"difficulty": "review-required",
|
|
||||||
"prerequisites": [],
|
|
||||||
"deliverables": ["project artifact"],
|
|
||||||
})
|
|
||||||
projects = {"projects": project_items}
|
|
||||||
rubrics = {"rubrics": [{"id": "draft-rubric", "title": "Draft Rubric", "criteria": ["correctness", "explanation"]}]}
|
|
||||||
attribution = {"source_name": course.source_name, "source_url": course.source_url, "rights_note": course.rights_note}
|
|
||||||
return DraftPack(pack=pack, concepts=concepts_yaml, roadmap=roadmap, projects=projects, rubrics=rubrics, review_report=review_flags, attribution=attribution)
|
|
||||||
|
|
||||||
|
|
||||||
def write_draft_pack(pack: DraftPack, outdir: str | Path) -> None:
|
|
||||||
out = Path(outdir)
|
|
||||||
out.mkdir(parents=True, exist_ok=True)
|
|
||||||
(out / "pack.yaml").write_text(yaml.safe_dump(pack.pack, sort_keys=False), encoding="utf-8")
|
|
||||||
(out / "concepts.yaml").write_text(yaml.safe_dump(pack.concepts, sort_keys=False), encoding="utf-8")
|
|
||||||
(out / "roadmap.yaml").write_text(yaml.safe_dump(pack.roadmap, sort_keys=False), encoding="utf-8")
|
|
||||||
(out / "projects.yaml").write_text(yaml.safe_dump(pack.projects, sort_keys=False), encoding="utf-8")
|
|
||||||
(out / "rubrics.yaml").write_text(yaml.safe_dump(pack.rubrics, sort_keys=False), encoding="utf-8")
|
|
||||||
review_lines = ["# Review Report", ""] + [f"- {flag}" for flag in pack.review_report] if pack.review_report else ["# Review Report", "", "- none"]
|
|
||||||
(out / "review_report.md").write_text("\n".join(review_lines), encoding="utf-8")
|
|
||||||
(out / "license_attribution.json").write_text(json.dumps(pack.attribution, indent=2), encoding="utf-8")
|
|
||||||
|
|
@ -1,83 +0,0 @@
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from dataclasses import dataclass, field
|
|
||||||
from typing import Callable
|
|
||||||
from .course_schema import NormalizedCourse, ConceptCandidate
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class RuleContext:
|
|
||||||
course: NormalizedCourse
|
|
||||||
concepts: list[ConceptCandidate]
|
|
||||||
review_flags: list[str] = field(default_factory=list)
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class Rule:
|
|
||||||
name: str
|
|
||||||
predicate: Callable[[RuleContext], bool]
|
|
||||||
action: Callable[[RuleContext], None]
|
|
||||||
|
|
||||||
|
|
||||||
def order_based_prerequisite_rule(context: RuleContext) -> None:
|
|
||||||
concept_titles = {c.title: c for c in context.concepts}
|
|
||||||
previous = None
|
|
||||||
for module in context.course.modules:
|
|
||||||
for lesson in module.lessons:
|
|
||||||
current = concept_titles.get(lesson.title)
|
|
||||||
if current is not None and previous is not None and previous.id not in current.prerequisites:
|
|
||||||
current.prerequisites.append(previous.id)
|
|
||||||
if current is not None:
|
|
||||||
previous = current
|
|
||||||
|
|
||||||
|
|
||||||
def duplicate_term_merge_rule(context: RuleContext) -> None:
|
|
||||||
seen = {}
|
|
||||||
deduped = []
|
|
||||||
for concept in context.concepts:
|
|
||||||
key = concept.title.strip().lower()
|
|
||||||
if key in seen:
|
|
||||||
seen[key].source_modules.extend(x for x in concept.source_modules if x not in seen[key].source_modules)
|
|
||||||
seen[key].source_lessons.extend(x for x in concept.source_lessons if x not in seen[key].source_lessons)
|
|
||||||
if concept.description and len(seen[key].description) < len(concept.description):
|
|
||||||
seen[key].description = concept.description
|
|
||||||
else:
|
|
||||||
seen[key] = concept
|
|
||||||
deduped.append(concept)
|
|
||||||
context.concepts[:] = deduped
|
|
||||||
|
|
||||||
|
|
||||||
def project_detection_rule(context: RuleContext) -> None:
|
|
||||||
for module in context.course.modules:
|
|
||||||
joined = " ".join(lesson.body for lesson in module.lessons).lower()
|
|
||||||
if "project" in joined or "capstone" in joined:
|
|
||||||
context.review_flags.append(f"Module '{module.title}' appears to contain project-like material; review project extraction.")
|
|
||||||
|
|
||||||
|
|
||||||
def review_flag_rule(context: RuleContext) -> None:
|
|
||||||
for module in context.course.modules:
|
|
||||||
if not any(lesson.exercises for lesson in module.lessons):
|
|
||||||
context.review_flags.append(f"Module '{module.title}' has no explicit exercises; mastery signals may be weak.")
|
|
||||||
for concept in context.concepts:
|
|
||||||
if not concept.mastery_signals:
|
|
||||||
context.review_flags.append(f"Concept '{concept.title}' has no extracted mastery signals; review manually.")
|
|
||||||
|
|
||||||
|
|
||||||
def build_default_rules(enable_prereq=True, enable_merge=True, enable_projects=True, enable_review=True) -> list[Rule]:
|
|
||||||
rules = []
|
|
||||||
if enable_prereq:
|
|
||||||
rules.append(Rule("order_based_prerequisite_rule", lambda ctx: True, order_based_prerequisite_rule))
|
|
||||||
if enable_merge:
|
|
||||||
rules.append(Rule("duplicate_term_merge_rule", lambda ctx: True, duplicate_term_merge_rule))
|
|
||||||
if enable_projects:
|
|
||||||
rules.append(Rule("project_detection_rule", lambda ctx: True, project_detection_rule))
|
|
||||||
if enable_review:
|
|
||||||
rules.append(Rule("review_flag_rule", lambda ctx: True, review_flag_rule))
|
|
||||||
return rules
|
|
||||||
|
|
||||||
|
|
||||||
def run_rules(context: RuleContext, rules: list[Rule]) -> RuleContext:
|
|
||||||
for rule in rules:
|
|
||||||
if rule.predicate(context):
|
|
||||||
rule.action(context)
|
|
||||||
return context
|
|
||||||
|
|
@ -1,26 +0,0 @@
|
||||||
from didactopus.course_ingest import parse_markdown_course, extract_concept_candidates
|
|
||||||
|
|
||||||
SAMPLE = '''
|
|
||||||
# Sample Course
|
|
||||||
|
|
||||||
## Module 1
|
|
||||||
### Lesson A
|
|
||||||
- Objective: Explain Topic A.
|
|
||||||
- Exercise: Do task A.
|
|
||||||
Topic A body.
|
|
||||||
|
|
||||||
### Lesson B
|
|
||||||
- Objective: Explain Topic B.
|
|
||||||
Topic B body.
|
|
||||||
'''
|
|
||||||
|
|
||||||
def test_parse_markdown_course() -> None:
|
|
||||||
course = parse_markdown_course(SAMPLE, "Sample Course")
|
|
||||||
assert course.title == "Sample Course"
|
|
||||||
assert len(course.modules) == 1
|
|
||||||
assert len(course.modules[0].lessons) == 2
|
|
||||||
|
|
||||||
def test_extract_concepts() -> None:
|
|
||||||
course = parse_markdown_course(SAMPLE, "Sample Course")
|
|
||||||
concepts = extract_concept_candidates(course)
|
|
||||||
assert len(concepts) >= 2
|
|
||||||
|
|
@ -1,43 +0,0 @@
|
||||||
from pathlib import Path
|
|
||||||
import json
|
|
||||||
|
|
||||||
from didactopus.agentic_loop import run_demo_agentic_loop
|
|
||||||
from didactopus.mastery_ledger import (
|
|
||||||
build_capability_profile,
|
|
||||||
export_capability_profile_json,
|
|
||||||
export_capability_report_markdown,
|
|
||||||
export_artifact_manifest,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def test_build_capability_profile() -> None:
|
|
||||||
state = run_demo_agentic_loop([
|
|
||||||
"foundations-statistics::descriptive-statistics",
|
|
||||||
"bayes-extension::prior",
|
|
||||||
])
|
|
||||||
profile = build_capability_profile(state, "Bayesian inference")
|
|
||||||
assert profile.domain == "Bayesian inference"
|
|
||||||
assert len(profile.artifacts) == 2
|
|
||||||
|
|
||||||
|
|
||||||
def test_exports(tmp_path: Path) -> None:
|
|
||||||
state = run_demo_agentic_loop([
|
|
||||||
"foundations-statistics::descriptive-statistics",
|
|
||||||
"bayes-extension::prior",
|
|
||||||
])
|
|
||||||
profile = build_capability_profile(state, "Bayesian inference")
|
|
||||||
|
|
||||||
json_path = tmp_path / "capability_profile.json"
|
|
||||||
md_path = tmp_path / "capability_report.md"
|
|
||||||
manifest_path = tmp_path / "artifact_manifest.json"
|
|
||||||
|
|
||||||
export_capability_profile_json(profile, str(json_path))
|
|
||||||
export_capability_report_markdown(profile, str(md_path))
|
|
||||||
export_artifact_manifest(profile, str(manifest_path))
|
|
||||||
|
|
||||||
assert json_path.exists()
|
|
||||||
assert md_path.exists()
|
|
||||||
assert manifest_path.exists()
|
|
||||||
|
|
||||||
data = json.loads(json_path.read_text(encoding="utf-8"))
|
|
||||||
assert data["domain"] == "Bayesian inference"
|
|
||||||
|
|
@ -1,24 +0,0 @@
|
||||||
from pathlib import Path
|
|
||||||
from didactopus.course_ingest import parse_markdown_course, extract_concept_candidates
|
|
||||||
from didactopus.rule_policy import RuleContext, build_default_rules, run_rules
|
|
||||||
from didactopus.pack_emitter import build_draft_pack, write_draft_pack
|
|
||||||
|
|
||||||
SAMPLE = '''
|
|
||||||
# Sample Course
|
|
||||||
|
|
||||||
## Module 1
|
|
||||||
### Lesson A
|
|
||||||
- Objective: Explain Topic A.
|
|
||||||
- Exercise: Do task A.
|
|
||||||
Topic A body.
|
|
||||||
'''
|
|
||||||
|
|
||||||
def test_emit_pack(tmp_path: Path) -> None:
|
|
||||||
course = parse_markdown_course(SAMPLE, "Sample Course")
|
|
||||||
concepts = extract_concept_candidates(course)
|
|
||||||
ctx = RuleContext(course=course, concepts=concepts)
|
|
||||||
run_rules(ctx, build_default_rules())
|
|
||||||
draft = build_draft_pack(course, ctx.concepts, "Tester", "REVIEW", ctx.review_flags)
|
|
||||||
write_draft_pack(draft, tmp_path)
|
|
||||||
assert (tmp_path / "pack.yaml").exists()
|
|
||||||
assert (tmp_path / "review_report.md").exists()
|
|
||||||
|
|
@ -1,24 +0,0 @@
|
||||||
from didactopus.course_ingest import parse_markdown_course, extract_concept_candidates
|
|
||||||
from didactopus.rule_policy import RuleContext, build_default_rules, run_rules
|
|
||||||
|
|
||||||
SAMPLE = '''
|
|
||||||
# Sample Course
|
|
||||||
|
|
||||||
## Module 1
|
|
||||||
### Lesson A
|
|
||||||
- Objective: Explain Topic A.
|
|
||||||
- Exercise: Do task A.
|
|
||||||
Topic A body.
|
|
||||||
|
|
||||||
### Lesson B
|
|
||||||
- Objective: Explain Topic B.
|
|
||||||
- Exercise: Do task B.
|
|
||||||
Topic B body.
|
|
||||||
'''
|
|
||||||
|
|
||||||
def test_rules_run() -> None:
|
|
||||||
course = parse_markdown_course(SAMPLE, "Sample Course")
|
|
||||||
concepts = extract_concept_candidates(course)
|
|
||||||
ctx = RuleContext(course=course, concepts=concepts)
|
|
||||||
run_rules(ctx, build_default_rules())
|
|
||||||
assert len(ctx.concepts) >= 2
|
|
||||||
Loading…
Reference in New Issue