xournal-joplin

Handwriting OCR for Xournal++ notes, built to eventually bridge into Joplin as well. Started from a simple question — could libzinnia-style handwriting recognition work as a plugin for either app? — and landed on a different architecture: a shared local OCR service both apps' plugin sandboxes can actually reach, plus a native Xournal++ plugin that talks to it.

Components

• ocr-daemon — a local HTTP daemon that runs TrOCR (via Transformers.js) for handwriting recognition, plus its own line-segmentation step so a whole page can be posted at once. Neither Xournal++'s Lua plugin API nor Joplin's sandboxed plugin runtime can load a native ONNX runtime directly, so this exists as the one shared piece both can call over HTTP (or a Unix socket — see below). Also runs in Docker with optional Bearer-token-authenticated network listeners, for use beyond just "something else on this machine."
• xournal-plugin — a native Xournal++ Lua plugin. Exports the current page, posts it to ocr-daemon, and lets you insert the result as a real text box or save it to a .txt file for something like Joplin to import.
• ocr-tray — a GTK system tray app for recognizing handwriting from any clipboard image (e.g. a screenshot) or file, not just from Xournal++. Not sandboxed, so it talks to ocr-daemon over plain TCP; puts the result on the clipboard and shows it in a selectable window.
• Joplin plugin — not built yet.

Screenshots

xournal-plugin: Plugins menu	xournal-plugin: recognized text	ocr-tray: menu

Quick start

cd ocr-daemon
npm install
npm start

That runs it in the foreground for trying it out. For something that sticks around, see ocr-daemon/README.md for running it as a plain ocr-daemon command (npm link) or as a systemd user service (npm run install:systemd).

Then install the Xournal++ plugin per xournal-plugin/README.md and try Recognize Handwriting (current page) from the Plugins menu.

Both ocr-daemon and ocr-tray also have .deb and AppImage builds — see ocr-daemon/README.md and ocr-tray/README.md. If you install ocr-tray's .deb after already using ./install-autostart.sh from a checkout, remove the per-checkout ~/.config/autostart/ocr-tray.desktop first — otherwise both autostart entries fire at login.

For running ocr-daemon somewhere other than "on the same machine as its callers" — a server, a container, exposed to a LAN — see ocr-daemon/README.md for the config-file-driven authenticated listeners, its Docker section, and .github/workflows/ocr-daemon-docker.yml for a working build/run/publish pipeline driven by GitHub Actions Variables and Secrets.

Why this shape

A few things that weren't obvious going in, each covered in more depth in the relevant README:

• TrOCR only recognizes one line at a time. Feed it a whole page and it silently collapses everything into one short, usually-wrong guess — ocr-daemon does its own line segmentation so callers can just post a full page.
• Ruled/dot-grid paper can get misread as text lines. ocr-daemon's segmenter is tuned against a real dot-grid export; the Xournal++ plugin sidesteps the problem entirely by exporting ink-only (background="none").
• A flatpak-sandboxed Xournal++ has no network access at all — not even to 127.0.0.1 — but does have full host filesystem access. ocr-daemon listens on a Unix socket for exactly this reason; see ocr-daemon/README.md and xournal-plugin/README.md.
• app.openDialog text can't be selected or copied, and there's no clipboard-set API. Confirmed against Xournal++'s own source, and clipboard-set-via-helper-process doesn't work around it under flatpak+Wayland either (tested three ways). The plugin instead offers inserting real (selectable) text or writing straight to a file.
• GTK3's clipboard API can't see images GNOME Shell's screenshot tool puts on the clipboard, at least on this Wayland setup — confirmed by comparing Gtk.Clipboard.wait_for_image() (nothing) against wl-paste (reads it fine) on the same clipboard state. ocr-tray uses wl-clipboard directly for both reading and writing instead; see ocr-tray/README.md.
• Publishing ocr-daemon's default Docker port does nothing at all. It's bound to 127.0.0.1 inside the container's own network namespace, which -p mapping can't reach — confirmed unreachable even hitting the container's real internal IP directly. Exposing anything from a container requires an explicit, authenticated 0.0.0.0 listener via a config file; see ocr-daemon/README.md.

Status

Early and personal-use-shaped rather than polished — accuracy is decent but not perfect (small model, CPU inference), text placement in the Xournal++ document is approximate, and there's no Joplin-side piece yet. Working end to end for the core loop: handwriting → recognized text → usable output, both from inside Xournal++ (xournal-plugin) and from any clipboard image or file (ocr-tray).

License

MIT