Compare commits
No commits in common. "e70460404e2769a45a5060afc9334dfb030ed326" and "24d91bd7559352bdc9248688087f4bb40b304846" have entirely different histories.
e70460404e
...
24d91bd755
277
README.md
277
README.md
|
|
@ -1,275 +1,10 @@
|
||||||
# PyOrgPatcher
|
# PyOrgPatcher
|
||||||
|
|
||||||
**PyOrgPatcher** is a lightweight command-line utility and Python library for **programmatically editing, extracting, and synchronizing parts of Org-mode documents** — all without Emacs.
|
PyOrgPatcher is a Python command-line utility that aid in workflows involving Emacs Org-Mode documents, as used in the PolyPaper project.
|
||||||
|
|
||||||
It provides a simple way to patch Org files from scripts, CI/CD jobs, or AI agents, making it ideal for:
|
Functions:
|
||||||
- Collaborators who work with Org files but don’t use Emacs.
|
- List Org-Mode elements in an Org-Mode file that can be modified or extracted.
|
||||||
- Reproducible research pipelines using literate programming.
|
- Replace the content of an Org-Mode element with text from STDIN or from a file.
|
||||||
- AI systems (e.g., LangChain, LangGraph, or MCP agents) that maintain structured text and code artifacts.
|
- With a JSON configuration, either extract multiple Org-Mode elements to associated files (without noweb substitution) or update an Org-Mode document's specified elements with the contents of files.
|
||||||
|
- Dry-run capability.
|
||||||
All components are under:
|
|
||||||
|
|
||||||
```
|
|
||||||
|
|
||||||
./code/
|
|
||||||
├── orgpatch.py # main tool
|
|
||||||
├── example.org # demonstration Org file
|
|
||||||
└── test_orgpatch.py # pytest unit tests
|
|
||||||
|
|
||||||
````
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🔧 Installation & Setup
|
|
||||||
|
|
||||||
### 1. Requirements
|
|
||||||
|
|
||||||
- **Python 3.8+**
|
|
||||||
- **pytest** (for testing)
|
|
||||||
|
|
||||||
Install dependencies:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python3 -m venv venv
|
|
||||||
source venv/bin/activate
|
|
||||||
pip install pytest
|
|
||||||
````
|
|
||||||
|
|
||||||
### 2. Clone or Copy the Tool
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git clone https://git,cns.fyi/welsberr/PyOrgPatcher.git
|
|
||||||
cd PyOrgPatcher/code
|
|
||||||
```
|
|
||||||
|
|
||||||
You can also copy just `orgpatch.py` to any project — it’s self-contained and requires no external libraries.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🧩 Example Org File
|
|
||||||
|
|
||||||
[`example.org`](./example.org) demonstrates the types of elements PyOrgPatcher understands:
|
|
||||||
|
|
||||||
```org
|
|
||||||
* Main Title
|
|
||||||
** Introduction
|
|
||||||
Some intro text.
|
|
||||||
|
|
||||||
#+NAME: intro-para
|
|
||||||
This is a named paragraph that will be replaced.
|
|
||||||
|
|
||||||
** Data
|
|
||||||
#+NAME: mytable
|
|
||||||
| Item | Value |
|
|
||||||
|------+-------|
|
|
||||||
| A | 1 |
|
|
||||||
| B | 2 |
|
|
||||||
|
|
||||||
** Code
|
|
||||||
#+NAME: code-snippet
|
|
||||||
#+BEGIN_SRC python
|
|
||||||
print("hello world")
|
|
||||||
#+END_SRC
|
|
||||||
|
|
||||||
** My Section
|
|
||||||
This section body will be replaced.
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🚀 Basic Command-Line Usage
|
|
||||||
|
|
||||||
List available patch targets:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./orgpatch.py list example.org
|
|
||||||
```
|
|
||||||
|
|
||||||
Replace a section body:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./orgpatch.py replace example.org --section "** My Section" --from section.txt
|
|
||||||
```
|
|
||||||
|
|
||||||
Replace a named block or table:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./orgpatch.py replace example.org --name code-snippet --from code.py
|
|
||||||
```
|
|
||||||
|
|
||||||
Or from standard input:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
printf "| X | 42 |\n" | ./orgpatch.py replace example.org --name mytable --stdin
|
|
||||||
```
|
|
||||||
|
|
||||||
Use `--backup` to create `example.org.bak`.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🔄 JSON Sync Mode
|
|
||||||
|
|
||||||
Batch operations for multiple elements are defined via JSON mappings.
|
|
||||||
|
|
||||||
### Example: `mapping.json`
|
|
||||||
|
|
||||||
```json
|
|
||||||
[
|
|
||||||
{"name": "code-snippet", "file": "code.py"},
|
|
||||||
{"name": "mytable", "file": "table.org"},
|
|
||||||
{"section": "** My Section", "file": "section.txt"}
|
|
||||||
]
|
|
||||||
```
|
|
||||||
|
|
||||||
### Apply Updates *into* the Org File (files → Org)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./orgpatch.py sync example.org --map mapping.json --direction in --backup
|
|
||||||
```
|
|
||||||
|
|
||||||
### Export Org Elements *to* Files (Org → files)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./orgpatch.py sync example.org --map mapping.json --direction out --mkdirs
|
|
||||||
```
|
|
||||||
|
|
||||||
Options:
|
|
||||||
|
|
||||||
* `--dry-run` — preview only
|
|
||||||
* `--backup` — create a backup before modifying
|
|
||||||
* `--mkdirs` — create destination folders automatically
|
|
||||||
* `--no-overwrite` — skip existing files on export
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🧪 Running Unit Tests
|
|
||||||
|
|
||||||
A complete [pytest](https://docs.pytest.org) test suite is included:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
pytest -q
|
|
||||||
```
|
|
||||||
|
|
||||||
This validates:
|
|
||||||
|
|
||||||
* Parsing of headings and `#+NAME:` blocks
|
|
||||||
* Section replacement
|
|
||||||
* Named block/table/paragraph replacement
|
|
||||||
* JSON mapping synchronization (`sync in` and `sync out`)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🧠 Integrating with AI Agent Tooling
|
|
||||||
|
|
||||||
### Why It Matters
|
|
||||||
|
|
||||||
Org-mode provides a **hierarchical, semantic structure** perfect for literate programming, documentation, and multi-language code notebooks — but until now, most AI systems couldn’t safely manipulate it without Emacs or brittle regex hacks.
|
|
||||||
|
|
||||||
**PyOrgPatcher** changes that:
|
|
||||||
It gives AI agents and LLM-based systems a structured interface to read, patch, and synchronize `.org` files, **treating them as composable data artifacts**.
|
|
||||||
|
|
||||||
### Example: Agent Workflow
|
|
||||||
|
|
||||||
Imagine an **AI documentation agent** that:
|
|
||||||
|
|
||||||
1. Reads `mapping.json` to identify relevant files and blocks.
|
|
||||||
2. Rebuilds code snippets or analyses (e.g., retraining results, data summaries).
|
|
||||||
3. Calls `orgpatch.py` (or its Python API) to update the corresponding Org blocks.
|
|
||||||
|
|
||||||
This lets an AI system manage “literate” files safely without needing to regenerate the entire Org document.
|
|
||||||
|
|
||||||
### In LangChain / LangGraph
|
|
||||||
|
|
||||||
Within a LangChain or LangGraph workflow, you can use `PyOrgPatcher` as a tool node:
|
|
||||||
|
|
||||||
```python
|
|
||||||
from langchain.tools import tool
|
|
||||||
import subprocess, json
|
|
||||||
|
|
||||||
@tool("update_org_block")
|
|
||||||
def update_org_block(name: str, content: str):
|
|
||||||
"""Update a named Org-mode block in example.org."""
|
|
||||||
cmd = [
|
|
||||||
"python3", "orgpatch.py", "replace", "example.org",
|
|
||||||
"--name", name, "--stdin"
|
|
||||||
]
|
|
||||||
subprocess.run(cmd, input=content.encode("utf-8"), check=True)
|
|
||||||
return f"Updated block '{name}' successfully."
|
|
||||||
```
|
|
||||||
|
|
||||||
Agents can then call this tool directly:
|
|
||||||
|
|
||||||
```python
|
|
||||||
update_org_block("analysis-results", "Mean=42\nStd=7.5\n")
|
|
||||||
```
|
|
||||||
|
|
||||||
Similarly, a chain could:
|
|
||||||
|
|
||||||
* Use `sync in` to load generated files into an Org document.
|
|
||||||
* Use `sync out` to extract tables or code for downstream tasks (e.g., testing, publishing, or compiling).
|
|
||||||
|
|
||||||
### Multi-Component Patch (MCP) Integration
|
|
||||||
|
|
||||||
In MCP-style systems — where AI agents patch **specific parts of multi-file projects** —
|
|
||||||
PyOrgPatcher acts as the **Org-layer patcher**:
|
|
||||||
|
|
||||||
* `sync in` = inject AI-generated code, results, or summaries.
|
|
||||||
* `sync out` = extract training data, code snippets, or narrative text.
|
|
||||||
* `replace` = fine-grained control of a single element in context.
|
|
||||||
|
|
||||||
This makes it ideal for **AI-assisted research notebooks**, **literate AI model reports**, and **collaborative documentation systems**.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🧰 Python Library Usage
|
|
||||||
|
|
||||||
PyOrgPatcher can also be used as a Python module:
|
|
||||||
|
|
||||||
```python
|
|
||||||
import orgpatch as op
|
|
||||||
|
|
||||||
lines = op.read_lines("example.org")
|
|
||||||
new_lines = op.replace_named(lines, "code-snippet", "print('updated via API')\n")
|
|
||||||
op.write_lines("example.org", new_lines)
|
|
||||||
```
|
|
||||||
|
|
||||||
This makes it easy to integrate into scripts or notebooks.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🧬 Example Use Case: Non-Emacs Collaboration
|
|
||||||
|
|
||||||
Researchers, engineers, or students who use VS Code, Sublime, or other editors can:
|
|
||||||
|
|
||||||
* Edit external files (`.py`, `.csv`, `.txt`) directly.
|
|
||||||
* Let an automated process update `main.org` via `sync in`.
|
|
||||||
* Keep the literate record synchronized without ever opening Emacs.
|
|
||||||
|
|
||||||
This lowers the barrier for collaboration in mixed-editor teams.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🏁 Summary
|
|
||||||
|
|
||||||
| Feature | Description |
|
|
||||||
| -------------------------- | ---------------------------------------------------- |
|
|
||||||
| **CLI + Library** | Command-line and Python module usage |
|
|
||||||
| **Emacs-Free Editing** | Safely patch `.org` files without Emacs |
|
|
||||||
| **JSON Batch Mode** | Bulk updates and exports |
|
|
||||||
| **AI/Automation Ready** | Ideal for LangChain, LangGraph, or MCP agents |
|
|
||||||
| **Fully Tested** | Comprehensive pytest suite included |
|
|
||||||
| **Collaboration-Friendly** | Perfect for teams mixing Org-mode with other editors |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**Author:** Wesley R. Elsberry
|
|
||||||
**Project:** PyOrgPatcher
|
|
||||||
**License:** MIT
|
|
||||||
**Location:** `./code/`
|
|
||||||
|
|
||||||
> “PyOrgPatcher bridges human-readable structure and machine-editable precision — empowering both collaborators and intelligent agents to work fluently with Org-mode documents.”
|
|
||||||
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,27 +0,0 @@
|
||||||
* Main Title
|
|
||||||
** Introduction
|
|
||||||
Some intro text.
|
|
||||||
|
|
||||||
#+NAME: intro-para
|
|
||||||
This is a named paragraph that will be replaced.
|
|
||||||
It continues until the first blank line.
|
|
||||||
|
|
||||||
** Data
|
|
||||||
#+NAME: mytable
|
|
||||||
| Item | Value |
|
|
||||||
|------+-------|
|
|
||||||
| A | 1 |
|
|
||||||
| B | 2 |
|
|
||||||
|
|
||||||
** Code
|
|
||||||
#+NAME: code-snippet
|
|
||||||
#+BEGIN_SRC python
|
|
||||||
print("hello world")
|
|
||||||
#+END_SRC
|
|
||||||
|
|
||||||
** My Section
|
|
||||||
This section body will be replaced.
|
|
||||||
It ends at the next heading of level ** or *.
|
|
||||||
|
|
||||||
** Another Section
|
|
||||||
Content of another section.
|
|
||||||
|
|
@ -1,18 +0,0 @@
|
||||||
[
|
|
||||||
{
|
|
||||||
"name": "code-snippet",
|
|
||||||
"file": "/mnt/data/orgpatch/code.py"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "mytable",
|
|
||||||
"file": "/mnt/data/orgpatch/tbl.org"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"section": "** My Section",
|
|
||||||
"file": "/mnt/data/orgpatch/my_section.txt"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "intro-para",
|
|
||||||
"file": "/mnt/data/orgpatch/intro.txt"
|
|
||||||
}
|
|
||||||
]
|
|
||||||
|
|
@ -1,18 +0,0 @@
|
||||||
[
|
|
||||||
{
|
|
||||||
"name": "code-snippet",
|
|
||||||
"file": "/mnt/data/orgpatch/exported/code.py"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "mytable",
|
|
||||||
"file": "/mnt/data/orgpatch/exported/table.org"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"section": "** My Section",
|
|
||||||
"file": "/mnt/data/orgpatch/exported/section.txt"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "intro-para",
|
|
||||||
"file": "/mnt/data/orgpatch/exported/intro.txt"
|
|
||||||
}
|
|
||||||
]
|
|
||||||
385
code/orgpatch.py
385
code/orgpatch.py
|
|
@ -1,385 +0,0 @@
|
||||||
#!/usr/bin/env python3
|
|
||||||
# orgpatch.py
|
|
||||||
"""
|
|
||||||
Replace parts of Org-mode files from the command line.
|
|
||||||
|
|
||||||
Supported targets:
|
|
||||||
1) Sections by exact heading text (e.g., "** My Section")
|
|
||||||
2) '#+NAME:'-labeled elements:
|
|
||||||
- Named blocks (#+BEGIN_... / #+END_...)
|
|
||||||
- Named tables (consecutive lines beginning with '|')
|
|
||||||
- Named paragraphs/lists immediately following '#+NAME:' (until blank line)
|
|
||||||
|
|
||||||
Also supports JSON "sync" to batch update/export:
|
|
||||||
- direction=in : update Org from external files
|
|
||||||
- direction=out : export from Org into external files (like tangle, but no noweb)
|
|
||||||
|
|
||||||
Usage:
|
|
||||||
List replaceable regions:
|
|
||||||
orgpatch.py list FILE.org
|
|
||||||
|
|
||||||
Replace a section body with stdin:
|
|
||||||
orgpatch.py replace FILE.org --section "** My Section" --stdin
|
|
||||||
|
|
||||||
Replace a named element with a file:
|
|
||||||
orgpatch.py replace FILE.org --name foo --from path.txt
|
|
||||||
|
|
||||||
Sync via JSON mapping (in: files → Org, out: Org → files):
|
|
||||||
orgpatch.py sync FILE.org --map mapping.json --direction in --backup
|
|
||||||
orgpatch.py sync FILE.org --map mapping.json --direction out --mkdirs
|
|
||||||
|
|
||||||
Mapping JSON format:
|
|
||||||
Either a top-level list:
|
|
||||||
[
|
|
||||||
{"name": "code-snippet", "file": "code.py"},
|
|
||||||
{"name": "mytable", "file": "table.org"},
|
|
||||||
{"section": "** My Section", "file": "section.txt"}
|
|
||||||
]
|
|
||||||
or an object with "entries": [ ...same as above... ]
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
import argparse
|
|
||||||
import re
|
|
||||||
import sys
|
|
||||||
import json
|
|
||||||
import shutil
|
|
||||||
import os
|
|
||||||
from typing import List, Tuple, Optional, Dict, Any
|
|
||||||
|
|
||||||
Heading = Dict[str, Any]
|
|
||||||
NamedElem = Dict[str, Any]
|
|
||||||
|
|
||||||
HEADING_RE = re.compile(r'^(\*+)\s+(.*)\s*$')
|
|
||||||
NAME_RE = re.compile(r'^\s*#\+NAME:\s*(\S+)\s*$', re.IGNORECASE)
|
|
||||||
BEGIN_RE = re.compile(r'^\s*#\+BEGIN_([A-Z0-9_]+)\b.*$', re.IGNORECASE)
|
|
||||||
END_RE_FMT = r'^\s*#\+END_{kind}\b.*$'
|
|
||||||
TABLE_RE = re.compile(r'^\s*\|')
|
|
||||||
|
|
||||||
def read_lines(path: str) -> List[str]:
|
|
||||||
with open(path, 'r', encoding='utf-8') as f:
|
|
||||||
return f.readlines()
|
|
||||||
|
|
||||||
def write_lines(path: str, lines: List[str]) -> None:
|
|
||||||
with open(path, 'w', encoding='utf-8') as f:
|
|
||||||
f.writelines(lines)
|
|
||||||
|
|
||||||
def parse_headings(lines: List[str]) -> List[Heading]:
|
|
||||||
heads: List[Heading] = []
|
|
||||||
for i, line in enumerate(lines):
|
|
||||||
m = HEADING_RE.match(line)
|
|
||||||
if m:
|
|
||||||
level = len(m.group(1))
|
|
||||||
title = m.group(2)
|
|
||||||
heads.append({'i': i, 'level': level, 'title': title})
|
|
||||||
return heads
|
|
||||||
|
|
||||||
def section_bounds(lines: List[str], headings: List[Heading], exact_heading: str) -> Optional[Tuple[int, int]]:
|
|
||||||
exact_heading = exact_heading.rstrip('\n')
|
|
||||||
start_idx = None
|
|
||||||
level = None
|
|
||||||
for h in headings:
|
|
||||||
heading_line_text = "{} {}".format('*' * h['level'], h['title'])
|
|
||||||
if heading_line_text == exact_heading:
|
|
||||||
start_idx = h['i']
|
|
||||||
level = h['level']
|
|
||||||
break
|
|
||||||
if start_idx is None:
|
|
||||||
return None
|
|
||||||
body_start = start_idx + 1
|
|
||||||
for h in headings:
|
|
||||||
if h['i'] <= start_idx:
|
|
||||||
continue
|
|
||||||
if h['level'] <= level:
|
|
||||||
return (body_start, h['i'])
|
|
||||||
return (body_start, len(lines))
|
|
||||||
|
|
||||||
def _scan_named_paragraph(lines: List[str], j: int) -> Optional[Tuple[int, int]]:
|
|
||||||
n = len(lines)
|
|
||||||
if j >= n:
|
|
||||||
return None
|
|
||||||
if BEGIN_RE.match(lines[j]) or TABLE_RE.match(lines[j]):
|
|
||||||
return None
|
|
||||||
if lines[j].strip() == "":
|
|
||||||
return None
|
|
||||||
k = j + 1
|
|
||||||
while k < n and lines[k].strip() != "":
|
|
||||||
k += 1
|
|
||||||
return (j, k)
|
|
||||||
|
|
||||||
def parse_named_elements(lines: List[str]) -> List[NamedElem]:
|
|
||||||
named: List[NamedElem] = []
|
|
||||||
i = 0
|
|
||||||
n = len(lines)
|
|
||||||
while i < n:
|
|
||||||
m = NAME_RE.match(lines[i])
|
|
||||||
if not m:
|
|
||||||
i += 1
|
|
||||||
continue
|
|
||||||
name = m.group(1)
|
|
||||||
j = i + 1
|
|
||||||
while j < n and lines[j].strip() == '':
|
|
||||||
j += 1
|
|
||||||
if j >= n:
|
|
||||||
i += 1
|
|
||||||
continue
|
|
||||||
mbeg = BEGIN_RE.match(lines[j])
|
|
||||||
if mbeg:
|
|
||||||
kind = mbeg.group(1).upper()
|
|
||||||
end_re = re.compile(END_RE_FMT.format(kind=re.escape(kind)), re.IGNORECASE)
|
|
||||||
k = j + 1
|
|
||||||
while k < n and not end_re.match(lines[k]):
|
|
||||||
k += 1
|
|
||||||
if k >= n:
|
|
||||||
content_start = j + 1
|
|
||||||
content_end = n
|
|
||||||
named.append({
|
|
||||||
'type': 'block',
|
|
||||||
'name': name,
|
|
||||||
'kind': kind,
|
|
||||||
'begin_i': j,
|
|
||||||
'end_i': n - 1,
|
|
||||||
'content_start': content_start,
|
|
||||||
'content_end': content_end
|
|
||||||
})
|
|
||||||
i = n
|
|
||||||
continue
|
|
||||||
named.append({
|
|
||||||
'type': 'block',
|
|
||||||
'name': name,
|
|
||||||
'kind': kind,
|
|
||||||
'begin_i': j,
|
|
||||||
'end_i': k,
|
|
||||||
'content_start': j + 1,
|
|
||||||
'content_end': k
|
|
||||||
})
|
|
||||||
i = k + 1
|
|
||||||
continue
|
|
||||||
if TABLE_RE.match(lines[j]):
|
|
||||||
k = j
|
|
||||||
while k < n and TABLE_RE.match(lines[k]):
|
|
||||||
k += 1
|
|
||||||
named.append({
|
|
||||||
'type': 'table',
|
|
||||||
'name': name,
|
|
||||||
'start_i': j,
|
|
||||||
'end_i': k
|
|
||||||
})
|
|
||||||
i = k
|
|
||||||
continue
|
|
||||||
para_bounds = _scan_named_paragraph(lines, j)
|
|
||||||
if para_bounds is not None:
|
|
||||||
s, e = para_bounds
|
|
||||||
named.append({
|
|
||||||
'type': 'para',
|
|
||||||
'name': name,
|
|
||||||
'start_i': s,
|
|
||||||
'end_i': e
|
|
||||||
})
|
|
||||||
i = e
|
|
||||||
continue
|
|
||||||
i = j
|
|
||||||
return named
|
|
||||||
|
|
||||||
def list_targets(lines: List[str]) -> None:
|
|
||||||
heads = parse_headings(lines)
|
|
||||||
print("== Sections ==")
|
|
||||||
for h in heads:
|
|
||||||
print(f" L{h['i']+1:>4} | level {h['level']} | {('*'*h['level'])} {h['title']}")
|
|
||||||
print("\n== #+NAME elements ==")
|
|
||||||
named = parse_named_elements(lines)
|
|
||||||
for e in named:
|
|
||||||
if e['type'] == 'block':
|
|
||||||
beg = e['begin_i'] + 1
|
|
||||||
end = e['end_i'] + 1
|
|
||||||
print(f" name={e['name']} | block {e['kind']} | lines {beg}-{end} (content {e['content_start']+1}-{e['content_end']})")
|
|
||||||
elif e['type'] == 'table':
|
|
||||||
print(f" name={e['name']} | table | lines {e['start_i']+1}-{e['end_i']}")
|
|
||||||
else:
|
|
||||||
print(f" name={e['name']} | paragraph | lines {e['start_i']+1}-{e['end_i']}")
|
|
||||||
|
|
||||||
def load_replacement(from_file: Optional[str], use_stdin: bool) -> str:
|
|
||||||
if from_file and use_stdin:
|
|
||||||
raise SystemExit("Choose either --from FILE or --stdin, not both.")
|
|
||||||
if from_file:
|
|
||||||
with open(from_file, 'r', encoding='utf-8') as f:
|
|
||||||
return f.read()
|
|
||||||
if use_stdin:
|
|
||||||
return sys.stdin.read()
|
|
||||||
raise SystemExit("You must provide replacement text with --from FILE or --stdin.")
|
|
||||||
|
|
||||||
def replace_section(lines: List[str], heading_text: str, new_body: str) -> List[str]:
|
|
||||||
heads = parse_headings(lines)
|
|
||||||
bounds = section_bounds(lines, heads, heading_text)
|
|
||||||
if not bounds:
|
|
||||||
raise SystemExit(f"Section not found: {heading_text!r}")
|
|
||||||
start, end = bounds
|
|
||||||
new_lines = [ln if ln.endswith('\n') else (ln + '\n') for ln in new_body.splitlines()]
|
|
||||||
return lines[:start] + new_lines + lines[end:]
|
|
||||||
|
|
||||||
def replace_named(lines: List[str], name: str, new_content: str) -> List[str]:
|
|
||||||
elems = parse_named_elements(lines)
|
|
||||||
target = next((e for e in elems if e['name'] == name), None)
|
|
||||||
if not target:
|
|
||||||
raise SystemExit(f"No '#+NAME: {name}' element found (supported: named blocks, tables, and paragraphs).")
|
|
||||||
if target['type'] == 'block':
|
|
||||||
c0, c1 = target['content_start'], target['content_end']
|
|
||||||
new_lines = [ln if ln.endswith('\n') else (ln + '\n') for ln in new_content.splitlines()]
|
|
||||||
return lines[:c0] + new_lines + lines[c1:]
|
|
||||||
elif target['type'] == 'table':
|
|
||||||
s, e = target['start_i'], target['end_i']
|
|
||||||
new_lines = [ln if ln.endswith('\n') else (ln + '\n') for ln in new_content.splitlines()]
|
|
||||||
for idx, ln in enumerate(new_lines, 1):
|
|
||||||
if ln.strip() and not ln.lstrip().startswith('|'):
|
|
||||||
print(f"Warning: replacement line {idx} for table '{name}' does not begin with '|'", file=sys.stderr)
|
|
||||||
return lines[:s] + new_lines + lines[e:]
|
|
||||||
else:
|
|
||||||
s, e = target['start_i'], target['end_i']
|
|
||||||
new_lines = [ln if ln.endswith('\n') else (ln + '\n') for ln in new_content.splitlines()]
|
|
||||||
if not new_lines or new_lines[-1].strip() != "":
|
|
||||||
new_lines.append("\n")
|
|
||||||
return lines[:s] + new_lines + lines[e:]
|
|
||||||
|
|
||||||
# ===== Sync helpers =====
|
|
||||||
|
|
||||||
def extract_named_content(lines: List[str], name: str) -> str:
|
|
||||||
elems = parse_named_elements(lines)
|
|
||||||
e = next((x for x in elems if x['name'] == name), None)
|
|
||||||
if not e:
|
|
||||||
raise SystemExit(f"No '#+NAME: {name}' element found.")
|
|
||||||
if e['type'] == 'block':
|
|
||||||
c0, c1 = e['content_start'], e['content_end']
|
|
||||||
return ''.join(lines[c0:c1])
|
|
||||||
elif e['type'] == 'table':
|
|
||||||
s, eidx = e['start_i'], e['end_i']
|
|
||||||
return ''.join(lines[s:eidx])
|
|
||||||
else:
|
|
||||||
s, eidx = e['start_i'], e['end_i']
|
|
||||||
return ''.join(lines[s:eidx])
|
|
||||||
|
|
||||||
def extract_section_body(lines: List[str], heading_text: str) -> str:
|
|
||||||
heads = parse_headings(lines)
|
|
||||||
bounds = section_bounds(lines, heads, heading_text)
|
|
||||||
if not bounds:
|
|
||||||
raise SystemExit(f"Section not found: {heading_text!r}")
|
|
||||||
s, e = bounds
|
|
||||||
return ''.join(lines[s:e])
|
|
||||||
|
|
||||||
def sync_apply_in(lines: List[str], mapping: list) -> List[str]:
|
|
||||||
"""Apply updates from files into the Org according to mapping entries."""
|
|
||||||
for idx, entry in enumerate(mapping, 1):
|
|
||||||
if 'file' not in entry:
|
|
||||||
raise SystemExit(f"Mapping entry #{idx} is missing 'file' field.")
|
|
||||||
fpath = entry['file']
|
|
||||||
try:
|
|
||||||
with open(fpath, 'r', encoding='utf-8') as fh:
|
|
||||||
content = fh.read()
|
|
||||||
except FileNotFoundError:
|
|
||||||
raise SystemExit(f"Mapping entry #{idx}: source file not found: {fpath!r}")
|
|
||||||
if 'section' in entry:
|
|
||||||
lines = replace_section(lines, entry['section'], content)
|
|
||||||
elif 'name' in entry:
|
|
||||||
lines = replace_named(lines, entry['name'], content)
|
|
||||||
else:
|
|
||||||
raise SystemExit(f"Mapping entry #{idx} must have either 'name' or 'section'.")
|
|
||||||
return lines
|
|
||||||
|
|
||||||
def sync_apply_out(lines: List[str], mapping: list, mkdirs: bool = False, overwrite: bool = True) -> None:
|
|
||||||
"""Extract content from Org into files according to mapping entries (does not alter Org)."""
|
|
||||||
for idx, entry in enumerate(mapping, 1):
|
|
||||||
if 'file' not in entry:
|
|
||||||
raise SystemExit(f"Mapping entry #{idx} is missing 'file' field.")
|
|
||||||
fpath = entry['file']
|
|
||||||
if mkdirs:
|
|
||||||
pathlib.Path(fpath).parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
if (not overwrite) and os.path.exists(fpath):
|
|
||||||
print(f"Skipping existing file (overwrite disabled): {fpath}", file=sys.stderr)
|
|
||||||
continue
|
|
||||||
if 'section' in entry:
|
|
||||||
text = extract_section_body(lines, entry['section'])
|
|
||||||
elif 'name' in entry:
|
|
||||||
text = extract_named_content(lines, entry['name'])
|
|
||||||
else:
|
|
||||||
raise SystemExit(f"Mapping entry #{idx} must have either 'name' or 'section'.")
|
|
||||||
with open(fpath, 'w', encoding='utf-8') as fh:
|
|
||||||
fh.write(text)
|
|
||||||
|
|
||||||
def load_mapping(path: str) -> list:
|
|
||||||
with open(path, 'r', encoding='utf-8') as fh:
|
|
||||||
data = json.load(fh)
|
|
||||||
if isinstance(data, dict) and 'entries' in data:
|
|
||||||
return data['entries']
|
|
||||||
if not isinstance(data, list):
|
|
||||||
raise SystemExit("Mapping JSON must be a list of entries or an object with an 'entries' list.")
|
|
||||||
return data
|
|
||||||
|
|
||||||
def main(argv: Optional[List[str]] = None) -> None:
|
|
||||||
p = argparse.ArgumentParser(description="Replace parts of Org files and sync via JSON map.")
|
|
||||||
sub = p.add_subparsers(dest='cmd', required=True)
|
|
||||||
|
|
||||||
p_list = sub.add_parser('list', help="List replaceable sections and #+NAME elements")
|
|
||||||
p_list.add_argument('file', help="Org file")
|
|
||||||
|
|
||||||
p_rep = sub.add_parser('replace', help="Replace a section body or a named element's contents")
|
|
||||||
p_rep.add_argument('file', help="Org file")
|
|
||||||
target = p_rep.add_mutually_exclusive_group(required=True)
|
|
||||||
target.add_argument('--section', help='Exact heading text to match (e.g., "** My Section")')
|
|
||||||
target.add_argument('--name', help="Name after '#+NAME:' to match")
|
|
||||||
src = p_rep.add_mutually_exclusive_group(required=True)
|
|
||||||
src.add_argument('--from', dest='from_file', metavar='FILE', help="Take replacement text from FILE")
|
|
||||||
src.add_argument('--stdin', action='store_true', help="Read replacement text from stdin")
|
|
||||||
p_rep.add_argument('--dry-run', action='store_true', help="Show the result to stdout without writing the file")
|
|
||||||
p_rep.add_argument('--backup', action='store_true', help="Write FILE.org.bak before modifying")
|
|
||||||
|
|
||||||
p_sync = sub.add_parser('sync', help="Sync blocks/sections using a JSON mapping (in: files → Org, out: Org → files)")
|
|
||||||
p_sync.add_argument('file', help='Org file')
|
|
||||||
p_sync.add_argument('--map', required=True, help='Path to JSON mapping file')
|
|
||||||
p_sync.add_argument('--direction', choices=['in','out'], required=True, help="'in' updates Org from files; 'out' exports from Org to files")
|
|
||||||
p_sync.add_argument('--dry-run', action='store_true', help='Preview Org result (direction=in) without writing file')
|
|
||||||
p_sync.add_argument('--backup', action='store_true', help='Write FILE.org.bak before modifying (direction=in only)')
|
|
||||||
p_sync.add_argument('--mkdirs', action='store_true', help='Create parent directories when exporting (direction=out)')
|
|
||||||
p_sync.add_argument('--no-overwrite', action='store_true', help='When exporting, do not overwrite existing files')
|
|
||||||
|
|
||||||
args = p.parse_args(argv)
|
|
||||||
lines = read_lines(args.file)
|
|
||||||
|
|
||||||
if args.cmd == 'list':
|
|
||||||
list_targets(lines)
|
|
||||||
return
|
|
||||||
|
|
||||||
if args.cmd == 'sync':
|
|
||||||
mapping = load_mapping(args.map)
|
|
||||||
if args.direction == 'in':
|
|
||||||
new_lines = sync_apply_in(lines, mapping)
|
|
||||||
if args.dry_run:
|
|
||||||
sys.stdout.write(''.join(new_lines))
|
|
||||||
return
|
|
||||||
if args.backup:
|
|
||||||
shutil.copyfile(args.file, args.file + '.bak')
|
|
||||||
write_lines(args.file, new_lines)
|
|
||||||
print(f"Updated {args.file} from {args.map}.")
|
|
||||||
else:
|
|
||||||
sync_apply_out(lines, mapping, mkdirs=args.mkdirs, overwrite=not args.no_overwrite)
|
|
||||||
print(f"Exported content from {args.file} to files per {args.map}.")
|
|
||||||
return
|
|
||||||
|
|
||||||
# replace
|
|
||||||
replacement = load_replacement(getattr(args, 'from_file', None), getattr(args, 'stdin', False))
|
|
||||||
if args.section:
|
|
||||||
new_lines = replace_section(lines, args.section, replacement)
|
|
||||||
else:
|
|
||||||
new_lines = replace_named(lines, args.name, replacement)
|
|
||||||
|
|
||||||
if args.dry_run:
|
|
||||||
sys.stdout.write(''.join(new_lines))
|
|
||||||
else:
|
|
||||||
if args.backup:
|
|
||||||
shutil.copyfile(args.file, args.file + ".bak")
|
|
||||||
write_lines(args.file, new_lines)
|
|
||||||
if args.section:
|
|
||||||
print(f"Replaced body of section {args.section!r} in {args.file}")
|
|
||||||
else:
|
|
||||||
print(f"Replaced contents of '#+NAME: {args.name}' in {args.file}")
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
|
||||||
main()
|
|
||||||
|
|
@ -1,130 +0,0 @@
|
||||||
import sys
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
# Import module from same folder
|
|
||||||
sys.path.insert(0, str(Path(__file__).parent))
|
|
||||||
import orgpatch as op
|
|
||||||
|
|
||||||
EXAMPLE_ORG = """* Main Title
|
|
||||||
** Introduction
|
|
||||||
Some intro text.
|
|
||||||
|
|
||||||
#+NAME: intro-para
|
|
||||||
This is a named paragraph that will be replaced.
|
|
||||||
It continues until the first blank line.
|
|
||||||
|
|
||||||
** Data
|
|
||||||
#+NAME: mytable
|
|
||||||
| Item | Value |
|
|
||||||
|------+-------|
|
|
||||||
| A | 1 |
|
|
||||||
| B | 2 |
|
|
||||||
|
|
||||||
** Code
|
|
||||||
#+NAME: code-snippet
|
|
||||||
#+BEGIN_SRC python
|
|
||||||
print("hello world")
|
|
||||||
#+END_SRC
|
|
||||||
|
|
||||||
** My Section
|
|
||||||
This section body will be replaced.
|
|
||||||
It ends at the next heading of level ** or *.
|
|
||||||
|
|
||||||
** Another Section
|
|
||||||
Content of another section.
|
|
||||||
"""
|
|
||||||
|
|
||||||
def write(p: Path, text: str):
|
|
||||||
p.write_text(text, encoding="utf-8")
|
|
||||||
|
|
||||||
def test_parse_headings_and_sections(tmp_path: Path):
|
|
||||||
org = tmp_path / "example.org"
|
|
||||||
write(org, EXAMPLE_ORG)
|
|
||||||
lines = op.read_lines(str(org))
|
|
||||||
heads = op.parse_headings(lines)
|
|
||||||
titles = [h['title'] for h in heads]
|
|
||||||
assert "Introduction" in titles
|
|
||||||
assert "My Section" in titles
|
|
||||||
b = op.section_bounds(lines, heads, "** My Section")
|
|
||||||
assert b is not None
|
|
||||||
s, e = b
|
|
||||||
body = ''.join(lines[s:e])
|
|
||||||
assert "This section body will be replaced." in body
|
|
||||||
|
|
||||||
def test_parse_named_elements(tmp_path: Path):
|
|
||||||
org = tmp_path / "example.org"
|
|
||||||
write(org, EXAMPLE_ORG)
|
|
||||||
lines = op.read_lines(str(org))
|
|
||||||
elems = op.parse_named_elements(lines)
|
|
||||||
names = {e['name']: e for e in elems}
|
|
||||||
assert names['intro-para']['type'] == 'para'
|
|
||||||
assert names['mytable']['type'] == 'table'
|
|
||||||
assert names['code-snippet']['type'] == 'block'
|
|
||||||
|
|
||||||
def test_replace_section(tmp_path: Path):
|
|
||||||
org = tmp_path / "example.org"
|
|
||||||
write(org, EXAMPLE_ORG)
|
|
||||||
lines = op.read_lines(str(org))
|
|
||||||
out = op.replace_section(lines, "** My Section", "New body\nMore\n")
|
|
||||||
joined = ''.join(out)
|
|
||||||
assert "New body" in joined
|
|
||||||
assert "This section body will be replaced." not in joined
|
|
||||||
|
|
||||||
def test_replace_named_block_table_para(tmp_path: Path):
|
|
||||||
org = tmp_path / "example.org"
|
|
||||||
write(org, EXAMPLE_ORG)
|
|
||||||
lines = op.read_lines(str(org))
|
|
||||||
|
|
||||||
out = op.replace_named(lines, "code-snippet", "print('updated')\n")
|
|
||||||
assert "print('updated')" in ''.join(out)
|
|
||||||
|
|
||||||
out2 = op.replace_named(out, "mytable", "| X | 9 |\n")
|
|
||||||
assert "| X | 9 |" in ''.join(out2)
|
|
||||||
|
|
||||||
out3 = op.replace_named(out2, "intro-para", "New intro\nSecond\n")
|
|
||||||
j3 = ''.join(out3)
|
|
||||||
assert "New intro" in j3
|
|
||||||
assert "named paragraph that will be replaced" not in j3
|
|
||||||
|
|
||||||
def test_sync_in_and_out(tmp_path: Path):
|
|
||||||
org = tmp_path / "example.org"
|
|
||||||
write(org, EXAMPLE_ORG)
|
|
||||||
|
|
||||||
code_src = tmp_path / "code.py"
|
|
||||||
tbl_src = tmp_path / "tbl.org"
|
|
||||||
sec_src = tmp_path / "section.txt"
|
|
||||||
para_src = tmp_path / "intro.txt"
|
|
||||||
write(code_src, "print('via in')\n")
|
|
||||||
write(tbl_src, "| Col | Val |\n|-----+-----|\n| A | 1 |\n")
|
|
||||||
write(sec_src, "Replaced via sync in.\nSecond line.\n")
|
|
||||||
write(para_src, "Intro via sync in.\n\n")
|
|
||||||
|
|
||||||
mapping = [
|
|
||||||
{"name": "code-snippet", "file": str(code_src)},
|
|
||||||
{"name": "mytable", "file": str(tbl_src)},
|
|
||||||
{"section": "** My Section", "file": str(sec_src)},
|
|
||||||
{"name": "intro-para", "file": str(para_src)},
|
|
||||||
]
|
|
||||||
|
|
||||||
lines = op.read_lines(str(org))
|
|
||||||
new_lines = op.sync_apply_in(lines, mapping)
|
|
||||||
out = ''.join(new_lines)
|
|
||||||
assert "print('via in')" in out
|
|
||||||
assert "| Col | Val |" in out
|
|
||||||
assert "Replaced via sync in." in out
|
|
||||||
assert "Intro via sync in." in out
|
|
||||||
|
|
||||||
export_dir = tmp_path / "exported"
|
|
||||||
export_dir.mkdir()
|
|
||||||
m_out = [
|
|
||||||
{"name": "code-snippet", "file": str(export_dir / "code.py")},
|
|
||||||
{"name": "mytable", "file": str(export_dir / "table.org")},
|
|
||||||
{"section": "** My Section", "file": str(export_dir / "section.txt")},
|
|
||||||
{"name": "intro-para", "file": str(export_dir / "intro.txt")},
|
|
||||||
]
|
|
||||||
op.sync_apply_out(new_lines, m_out, mkdirs=True, overwrite=True)
|
|
||||||
|
|
||||||
assert (export_dir / "code.py").read_text(encoding="utf-8").strip() == "print('via in')"
|
|
||||||
assert "| Col | Val |" in (export_dir / "table.org").read_text(encoding="utf-8")
|
|
||||||
assert "Replaced via sync in." in (export_dir / "section.txt").read_text(encoding="utf-8")
|
|
||||||
assert "Intro via sync in." in (export_dir / "intro.txt").read_text(encoding="utf-8")
|
|
||||||
Loading…
Reference in New Issue