# Daily Note to Longform Project Processor Automatically extracts research-relevant content from your daily notes and routes it to your Longform project research notes. ## What It Does 1. **Detects project mentions** in your daily notes (dictation-friendly!) 2. **Extracts fragments**: brainstorms, questions, citations, draft chunks 3. **Routes content** to project-specific research notes 4. **Preserves structure**: keeps headings, dates, links to original notes ## Quick Start ### Process Today's Note ```bash cd "/Users/nickyoung/My Obsidian Vault/obsidian-linker" ./process_today.sh ``` ### Process a Specific Note ```bash python3 daily_note_processor.py "/path/to/daily/note.md" ``` ### Test Mode (No File Changes) ```bash python3 daily_note_processor.py # Processes last 5 notes, shows what it would do ``` ## How It Detects Projects The system looks for: ### 1. **Natural Language Mentions** (dictation-friendly!) - "for the LLM aesthetics paper" - "thinking about text mechanics" - "relates to my Carlson project" - "in my work on environmental aesthetics" ### 2. **Keyword Clustering** If multiple project-related keywords appear together: - LLM + aesthetics + Carlson → LLM Aesthetics project - Environmental + aesthetics + AI → Environmental AI project ### 3. **Citation Patterns** Automatically detects: `Author (YEAR)` or `(Author YEAR, p. XX)` ## What Gets Extracted ### Brainstorms - Bullet point lists - Must be >20 characters to avoid capturing trivial lists ### Questions - Lines ending with `?` - Includes nested questions in bullet points ### Citations - Author/year format with surrounding context - Avoids duplicates ### Draft Chunks - Substantial paragraphs (>200 characters) - Prose sections that aren't bullet lists ## Where It Goes Content is routed to project research notes: ``` Writing/research/ ├── generative aesthetics of ai/ │ └── _Research Notes - llm_aesthetics.md └── environmental aesthetics of ai (short)/ └── _Research Notes - environmental_ai_short.md ``` Each research note has sections: - **Brainstorms** - **Questions** - **Citations & References** - **Draft Fragments** - **Related Ideas** ## Research Note Format ```markdown ### From [[2025-11-22]] - Section Heading [Your content here, with original structure preserved] ``` Everything links back to the original daily note for context. ## Adding New Projects Edit `daily_note_processor.py` and add to the `PROJECTS` dictionary: ```python "your_project_key": { "names": [ "Project name variations", "Keywords you might dictate" ], "folder": Path("/path/to/longform/project"), "keywords": ["key", "words", "to", "detect"] } ``` ## Current Projects 1. **llm_aesthetics** - LLM/AI aesthetics, text mechanics, Carlson 2. **environmental_ai_short** - Environmental aesthetics short paper ## Triage Unmatched substantial content goes to a "triage" list (shown in output). Review these periodically to see if they should be added to projects. ## Tips ### For Dictation Just naturally mention the project: - ✓ "for the text mechanics paper" - ✓ "thinking about Carlson's order appreciation" - ✗ No need for #tags or [[links]] ### Check What Was Extracted After running, open the research note: ``` Writing/research/[project]/_Research Notes - [project].md ``` ### Batch Processing Process multiple notes: ```bash for note in /path/to/Daily\ Notes/2025-*.md; do python3 daily_note_processor.py "$note" done ``` ## Next Steps This is Phase 1: **Project Detection & Content Extraction** Coming soon: - Weekly intelligence reports - Question surfacing - Citation tracking - Self-critique pattern analysis - Teaching content automation ## Troubleshooting **"No fragments detected"** - Check that your daily note has substantial content (not just template) - Make sure content isn't in excluded sections (diary, created, modified) **"Wrong project detected"** - Adjust keyword weights in `PROJECTS` dictionary - Add more specific project names **"Content not extracted"** - Check minimum length thresholds (bullets >20 chars, paragraphs >200 chars) - Ensure content isn't in commented sections (`%%...%%`) ## Files - `daily_note_processor.py` - Main processor - `process_today.sh` - Quick runner for today's note - `README-DailyNoteProcessor.md` - This file