127 lines
4.3 KiB
Markdown
127 lines
4.3 KiB
Markdown
# EcoSpecies Modernization Roadmap
|
|
|
|
## Target Product
|
|
|
|
Create a Docker Compose-based, open-source EcoSpecies successor that:
|
|
|
|
- ingests legacy SLH text files and future species submissions
|
|
- exposes a stable API for species, sections, citations, and ecological linkages
|
|
- provides a responsive public web app
|
|
- supports researcher/editor workflows for curation and publishing
|
|
- generates exports aligned with legacy reporting needs and future FLELMR-style outputs
|
|
|
|
## Recommended Stack
|
|
|
|
### Core platform
|
|
|
|
- Backend: Python API service
|
|
- Primary datastore: PostgreSQL
|
|
- Search/indexing: PostgreSQL full-text initially, optional Meilisearch/OpenSearch later
|
|
- Frontend: static SPA or React-based client once requirements stabilize
|
|
- Deployment/runtime: Docker Compose for development and small-scale deployment
|
|
|
|
### Why this stack
|
|
|
|
- permissive licenses
|
|
- strong support for text ingestion, APIs, and data processing
|
|
- easy local development
|
|
- clear path from prototype to production
|
|
|
|
## Product Capabilities By Phase
|
|
|
|
### Phase 0: Discovery and migration planning
|
|
|
|
- Inventory legacy assets and user-facing capabilities.
|
|
- Capture the replacement architecture and ingestion strategy.
|
|
- Define acknowledgements, provenance, and licensing boundaries.
|
|
|
|
### Phase 1: Ingestion foundation
|
|
|
|
- Parse legacy `.txt` SLH inputs into structured JSON records.
|
|
- Normalize common metadata: title, scientific name, common name, FLELMR code, headings, references.
|
|
- Create ingest diagnostics to flag malformed files and missing metadata.
|
|
|
|
### Phase 2: Public read experience
|
|
|
|
- Species listing and search.
|
|
- Species detail view with section navigation.
|
|
- Provenance and acknowledgement display.
|
|
- Summary metrics on corpus coverage.
|
|
|
|
### Phase 3: Structured persistence
|
|
|
|
- Move parsed content into PostgreSQL.
|
|
- Add editor-safe import jobs and audit metadata.
|
|
- Preserve raw source alongside normalized records.
|
|
- Establish authentication and role-based access for editor and admin workflows.
|
|
- Add persisted editorial workflow state for draft, review, and published records.
|
|
- Make document sections individually addressable for editor review and revision, with audit history for section-level changes.
|
|
|
|
### Phase 4: Linkages and visualization
|
|
|
|
- Model predator/prey, habitat, and ecological association edges.
|
|
- Add graph endpoints and species-relationship views.
|
|
- Support public-friendly visual explanations and expert filters.
|
|
|
|
### Phase 5: Reports and export
|
|
|
|
- Recreate legacy-like text/RTF export.
|
|
- Add machine-readable export formats such as JSON and Markdown.
|
|
- Support FLELMR-oriented authoring/export profiles.
|
|
|
|
### Phase 6: Assisted research workflows
|
|
|
|
- Add local-LLM-assisted extraction and drafting in a human-review loop.
|
|
- Integrate bibliography tooling for citation consolidation.
|
|
- Support candidate-species intake for records not yet in the historical corpus.
|
|
- Restrict assisted drafting and publication actions to authenticated editorial roles.
|
|
|
|
## Data Model Direction
|
|
|
|
Initial core entities:
|
|
|
|
- `species`
|
|
- `source_document`
|
|
- `document_section`
|
|
- `citation`
|
|
- `taxon`
|
|
- `linkage`
|
|
- `media_asset`
|
|
- `ingest_run`
|
|
|
|
Key design rules:
|
|
|
|
- preserve raw source text
|
|
- retain provenance and import timestamps
|
|
- separate public published records from draft/editor states
|
|
- make sections addressable for citation and graph linking
|
|
|
|
## LLM Extension Strategy
|
|
|
|
Use local models only for assistive tasks, never silent publication:
|
|
|
|
- extracting candidate structured fields from new SLH text
|
|
- suggesting missing headings or linkage labels
|
|
- clustering similar citations
|
|
- drafting summaries for editor review
|
|
|
|
Guardrails:
|
|
|
|
- raw text remains authoritative
|
|
- all generated content is marked as draft
|
|
- every automated extraction stores source spans where possible
|
|
|
|
## Development Roadmap
|
|
|
|
1. Implement a thin ingestion API over the legacy text corpus.
|
|
2. Build a responsive browser UI for listing and viewing species.
|
|
3. Add a persistent PostgreSQL-backed ingest store.
|
|
4. Introduce export and visualization services.
|
|
5. Add editorial workflows and local-LLM assistance.
|
|
|
|
## Definition Of Done For The Initial Milestone
|
|
|
|
- `docker compose up` starts a working API and frontend.
|
|
- The system can enumerate the legacy corpus and show parsed species detail for at least one real SLH file.
|
|
- Project docs describe the migration approach, target architecture, and next phases.
|