# Granola Sync

## What

A standalone script (`granola_sync.py`) that pulls meeting notes and full transcripts from Granola's official Enterprise API and writes one Markdown file per meeting into a local transcripts directory. It is incremental (only fetches new/changed notes), uses an API key (no scraping or local-app token extraction), and depends on nothing but the Python 3 standard library.

## Who

Authored by the pipeline maintainer. Consumed by the action extractor (which reads the Markdown files) and by any agent asked to rebuild the sync. The human only touches it to supply an API key once and to set the output directory.

## Why

Granola exposes an official read API (`public-api.granola.ai/v1`) gated by a workspace API key. Using it avoids the fragile alternative of reading the desktop app's local credential store, which is encrypted and changes between app versions. The API returns the **verbatim transcript** (speaker-attributed, timestamped) plus the AI summary — strictly richer than the summary-only notes the desktop app persists locally. The script keeps a small state file so that after the first full backfill it only fetches notes updated since the last run, keeping each run fast and well under the rate limit.

## Bounds

- **Not a transcriber.** It fetches notes that already exist; it does not record audio or generate transcripts.
- **Not a two-way sync.** Read-only. It never writes back to the provider.
- **Not scope-elevating.** It returns only what the API key's scope allows (e.g. notes the key's owner owns or that are shared with them). It cannot see notes the key cannot.
- **Not a general HTTP client.** It speaks exactly two endpoints: list notes and get note.
- **Not responsible for downstream parsing.** It defines the Markdown shape (see Interface); interpreting it is the extractor's job.

## Gotchas

- **The key scope determines coverage.** A "personal notes" key returns notes you own and notes shared with you; a "public/workspace" key returns workspace-shared notes. Pick the scope that matches what you want to capture; they are different sets.
- **First run vs incremental is decided by the state file.** Delete `sync_state.json` to force a full re-pull. With the state file present, only notes with a newer `updated_at` (or created after `last_run`) are fetched.
- **The window bounds what's fetched, not what's already on disk.** Setting `GRANOLA_SINCE` later does not delete previously-synced older files; it only stops fetching below the floor. Narrowing the window never removes data. To prune old transcripts, delete the files.
- **A floor plus incremental can still re-fetch an old, recently-edited note unless the floor is enforced.** `updated_after = last_run` would surface a 2-year-old note edited yesterday; the `created_at < GRANOLA_SINCE` skip is what keeps it out. Apply the floor in both paths.
- **`updated_after` catches edits, not just new notes.** A note edited after the last run is re-fetched and its file overwritten — intended.
- **Rate limit is real.** The API allows a few requests/second; the script paces calls and backs off on HTTP 429. Don't remove the delay.
- **Filenames embed a short id to avoid collisions.** Two meetings with the same title on the same day differ by the trailing `[shortid]`. Downstream code must treat the filename as opaque and read the frontmatter for identity.
- **iCloud/synced output dirs lag.** If the transcripts dir is a cloud-synced folder, files appear locally immediately but may take time to propagate to other devices. Not a sync-script bug.

## How

1. Resolve the API key: `GRANOLA_API_KEY` env, else read `${GS_CONFIG_DIR}/granola_api_key` (chmod 600).
2. Load state from `${GS_DATA_DIR}/granola_sync_state.json`.
3. Determine the window:
   - **First run (no state):** `created_after = GRANOLA_SINCE` if set, otherwise unbounded (full history).
   - **Incremental (state present):** `updated_after = last_run`.
   - In both cases apply `created_before = GRANOLA_UNTIL` if set, and treat `GRANOLA_SINCE` as a persistent floor: skip any note whose `created_at` is earlier than it.
   Paginate `GET /v1/notes?page_size=30&…&cursor=…` until `hasMore` is false, collecting `(id, updated_at, created_at)`.
4. For each note whose `updated_at` differs from state, `GET /v1/notes/{id}?include=transcript`.
5. Render Markdown (see Interface) and write to `${GS_TRANSCRIPTS_DIR}/{YYYY-MM-DD} - {safe title} [{shortid}].md`.
6. Record `synced[id] = updated_at`; set `last_run`; save state.

All requests send `Authorization: Bearer {key}`. Pace requests (~4/s) and retry 429s with exponential backoff.

## Example

A first run with a freshly minted personal-scope key, no state file: the script lists 312 notes across 11 pages, fetches each with `include=transcript`, and writes 312 files. A subsequent run two days later lists with `updated_after=<last_run>`, finds 4 notes (3 new meetings, 1 edited), writes/overwrites 4 files, and updates the state. A representative output file `2026-01-27 - Quarterly Review [a1b2c3d4].md` contains frontmatter (`granola_id: not_…`, `created_at: 2026-01-27T15:30:00Z`), an attendee line, a `## Summary` section, and a `## Transcript` section with lines like `**Me** [15:31:02] …` / `**Them** [15:31:08] …`.

## Calibration

- Base URL: `https://public-api.granola.ai/v1`.
- `PAGE_SIZE = 30` (API maximum).
- `REQ_DELAY = 0.25` seconds between calls (~4 req/s, under the ~5/s limit).
- `MAX_RETRIES = 5`, exponential backoff on 429/network errors.
- Sync window: `GRANOLA_SINCE` / `GRANOLA_UNTIL` (ISO dates) both unset by default → first run pulls full history. Set `GRANOLA_SINCE` to bound the backfill (e.g. last 90 days) and as a permanent floor. Incremental always uses `updated_after = last_run`.
- Speaker label mapping: source `microphone` → `Me`, `speaker` → `Them`.
- Output filename: `{date} - {title sanitised to [\w\s-], ≤80 chars} [{first 8 chars of id}].md`.

## Interface

**Provider API (exact field names).** Base `https://public-api.granola.ai/v1`, `Authorization: Bearer <key>`.

`GET /v1/notes` — query params: `created_before`, `created_after`, `updated_after` (ISO date or date-time), `cursor` (string), `page_size` (1–30, default 10). Response:
```json
{
  "notes": [
    { "id": "not_…", "object": "note", "title": "…",
      "owner": { "name": "…", "email": "…" },
      "created_at": "ISO8601", "updated_at": "ISO8601" }
  ],
  "hasMore": true,
  "cursor": "…|null"
}
```

`GET /v1/notes/{note_id}?include=transcript` — `note_id` matches `^not_[A-Za-z0-9]{14}$`. Response:
```json
{
  "id": "not_…", "object": "note", "title": "…|null",
  "owner": { "name": "…", "email": "…" },
  "created_at": "ISO8601", "updated_at": "ISO8601",
  "calendar_event": { "event_title": "…", "invitees": [{ "email": "…" }],
    "organiser": "…", "calendar_event_id": "…",
    "scheduled_start_time": "ISO8601", "scheduled_end_time": "ISO8601" },
  "attendees": [ { "name": "…", "email": "…" } ],
  "folder_membership": [ { "id": "fol_…", "object": "folder", "name": "…" } ],
  "summary_text": "…", "summary_markdown": "…|null",
  "transcript": [ { "speaker": { "source": "microphone|speaker" },
    "text": "…", "start_time": "ISO8601", "end_time": "ISO8601" } ] | null
}
```
Pagination: follow `cursor` while `hasMore` is true. Incremental key: a note is changed iff its `updated_at` differs from the stored value. `transcript` is `null` when a note has none; `summary_markdown` may be `null` (fall back to `summary_text`). Speaker `source` is exactly `microphone` (the key owner) or `speaker` (everyone else). Defensive note: treat these names as the contract, but a robust implementation isolates them in small field-accessor helpers so a future API rename is a one-line change.

Each transcript file MUST be:

```
---
granola_id: <provider id>
title: "<title>"
created_at: <ISO8601>
updated_at: <ISO8601>
owner: "<name> <email>"   # optional
---

# <title>

*<scheduled_start> - <scheduled_end>*            # optional

**Attendees:** <comma-separated names>           # optional

## Summary
<markdown summary>

## Transcript
**Me** [HH:MM:SS] <utterance>
**Them** [HH:MM:SS] <utterance>
...
```

State file `${GS_DATA_DIR}/granola_sync_state.json`:
```json
{ "synced": { "<note_id>": "<updated_at>" }, "last_run": "<ISO8601 Z>" }
```

CLI: `python3 granola_sync.py [--since YYYY-MM-DD] [--until YYYY-MM-DD] [--full]`. Flags override `GRANOLA_SINCE`/`GRANOLA_UNTIL`; `--full` ignores the state file and re-pulls the whole window. No required args. Exit non-zero only on unrecoverable auth failure.

## Acceptance

- With a valid key and empty state, a run creates one `.md` file per accessible note, each containing a `## Transcript` section with at least one speaker-attributed line for notes that have a transcript.
- A second immediate run writes zero files (nothing changed) and updates `last_run`.
- With `GRANOLA_SINCE` set, the first run fetches only notes created on/after that date; with `GRANOLA_UNTIL` also set, only notes within the window — and no note created before the floor is ever written, even on later incremental runs.
- Editing one note upstream then running re-writes exactly that one file.
- An invalid/expired key causes a clean exit with a clear message and no partial files.
- No transcript content is ever sent to any third party; the only outbound calls are to the provider API.

## Scope

Covers fetching and persisting transcripts as Markdown, and the sync state. Does not cover scheduling (see `PIPELINE_SCHEDULING_CONTEXT.md`) or any interpretation of transcript content (see `ACTION_EXTRACTION_CONTEXT.md`).

---

**Format:** https://contextcapsules.com/v2/CONTEXT_CAPSULES.md
**Sensitivity:** Public
**Status:** Active
**References:** ACTION_EXTRACTION_CONTEXT.md, PIPELINE_SCHEDULING_CONTEXT.md
**Owner:** Project maintainer
**Lineage:**
- 2026: Implemented against the official provider API after an earlier local-token approach proved fragile across app updates.
- 2026: Interface hardened with exact request params and response field names after a clean-room build had to infer them — closes the gap between "contract pinned" and "field names pinned".
- 2026: Added a configurable sync window (`GRANOLA_SINCE`/`GRANOLA_UNTIL`, `--since`/`--until`, `--full`) after a clean-room build surfaced that the spec only allowed full-history-or-incremental, with no bounded time range.