2 Commits

Author	SHA1	Message	Date
Jabber Developer	2952466abd	feat(history): consolidate logging controls and add dbbackend statusbar indicator ... Deprecate /logging command in favor of /history for chat logging control. /history off now stops persistence (sets PREF_DBLOG=off + PREF_CHLOG=false) in addition to hiding history on open. /history on restores persistence (re-enabling PREF_DBLOG=on if it was off) and PREF_CHLOG. statusbar - PREF_STATUSBAR_SHOW_DBBACKEND (default ON) gates the [sqlite] / [flatfile] indicator; toggle via "/statusbar show|hide dbbackend" /history off|on - "/history off" now stops persistence as well as hiding history on open: sets PREF_DBLOG=off + PREF_CHLOG=false in addition to PREF_HISTORY=false - "/history on" restores persistence (re-enabling PREF_DBLOG=on if it was off) and PREF_CHLOG, in addition to PREF_HISTORY=true /logging - Deprecated /logging command. It now prints a single notice pointing to /history; CMD_PREAMBLE trimmed (min_args=0, no setting_func, syntax/args/examples dropped); subcommand 'group' removed from autocomplete - All "use '/logging chat on' to enable" hints replaced with "/history on" across /omemo-log, /pgp-log, /otr-log, and /ox-log /privacy logging - Single-pass validation across {on, off, redact, flatfile} - Backend-switching values (on, flatfile) now take effect immediately when connected (via log_database_switch_backend), matching "/history switch" behaviour; off/redact only flip pref bits and keep the live backend open /correction off - win_print_outgoing and win_print_outgoing_with_receipt now gate _win_correct on PREF_CORRECTION_ALLOW, matching incoming-msg paths. Previously a peer's correction reflected via XEP-0280 carbons (or the user's own /correct invocation) was applied in-buffer regardless of the pref console - Drop orphaned /logging chat reference from cons_privacy_setting() - Delete cons_logging_setting() and remove its call from cons_show_log_prefs() autocomplete - Add dbbackend to statusbar_show_ac - Remove logging_ac entries for chat/group subcommands - Remove _logging_autocomplete() param handler for chat subcommand database_flatfile - Two local g_strndup allocations switched from char* with manual g_free to auto_gchar gchar* for automatic cleanup tests - Update test_cmd_otr.c to expect new /history-on warning messages @author: jabber.developer2 <jabber.developer2@jabber.space>	2026-05-16 15:33:42 +00:00
Jabber Developer	3f36c303c2	feat(history): flat-file backend with bidirectional SQLite migration ... A flat-file alternative to the SQLite chatlog backend with runtime switching, full migration tooling, integrity verification, and a synthetic load harness. SQLite remains the default; both backends share one dispatch layer (db_backend_t vtable) so callers don't change. Storage layout - Per-contact append-only `flatlog/<account>/<contact>/history.log` under XDG_DATA_HOME, one line per message - Single-line file header with embedded format-version marker (FLATFILE_FORMAT_VERSION); reader warns on missing or mismatched marker, writer and checker stay in sync via preprocessor stringification - Deterministic key=value metadata (`id`, `aid`, `corrects`, `to`, `to_res`, `read`) plus escaped body \u2014 `\|`, `\]`, `\\`, `\n`, `\r` literals prevent log injection - Sparse byte-offset index (FF_INDEX_STEP=500) per contact for O(log n) time-range lookups; rebuilt on inode / size / mtime change, extended in-place when the file just grew - Per-contact GHashTable caches for archive_id presence and stanza_id \u2192 from_jid mapping (O(1) MAM dedup, O(1) LMC sender validation) Hardening - Path-traversal protection: JID directory name normalisation (`@` \u2192 `_at_`, slashes and `..` rejected at construction); every per-contact path is anchored under the account's flatlog/ directory and validated before open - Symlink-attack protection: every fopen / open uses O_NOFOLLOW; on ELOOP the operation aborts with an error rather than following - Filesystem permissions: log files created with mode 0600, directories with mode 0700; both enforced at creation, verified on each open and reported on drift by `/history verify` - Atomic crash-safe export: write to a temp file via mkstemp (mode 0600, random suffix, no name collisions between concurrent exports), fsync, then rename \u2014 partial state never replaces the live file - Concurrency: advisory flock(LOCK_EX) held for the duration of every write, including append from live messages and full rewrite from export, so two profanity processes can't interleave bytes on the same log - DoS / abuse guards: * FF_MAX_LINE_LEN = 10 MB \u2014 lines longer than this are rejected at read with a warning; the parser will not allocate unbounded memory for a single record * FF_MAX_LMC_DEPTH = 100 \u2014 `corrects:` chain walk stops at this depth and emits a warning, preventing a malicious correction cycle from spinning the apply pass * FF_VERSION_SCAN_MAX = 16 \u2014 header version probe never reads past 16 leading comment lines, even on garbage input * Empty / inverted byte-range early-return in page-up read path so a malformed time filter cannot cause an unbounded scan * Zero-entry index guard so a file whose every line failed to parse cannot cause a NULL deref on later page-up - LMC sender validation: an incoming correction whose sender does not match the original message's sender is rejected at write time and surfaced via cons_show_error; a cycle in the apply pass is broken via a visited-set - jid_create_from_bare_and_resource treats NULL, empty string, and the literal "(null)" as no resource and returns a bare jid; similar normalisation for barejid eliminates the legacy "user@host/(null)" artefact that leaked into stored fulljids whenever g_strdup_printf("%s", NULL) ran inside create_fulljid Commands - `/history switch sqlite|flatfile` \u2014 runtime backend swap, closes the old backend and opens the new one without reconnecting - `/history export [<jid>]` \u2014 SQLite -> flat-file, merging with any existing flatlog (dedup keyed on a SHA-256 hash mixing stanza_id, timestamp, from_jid, body \u2014 robust against id reuse by older clients) - `/history import [<jid>]` \u2014 flat-file -> SQLite, same merge semantics, runs inside a single SQLite transaction with rollback on per-contact failure - `/history verify [<jid>]` \u2014 integrity check; emits a structured list of issues (ERROR / WARNING / INFO) per file: * file-level: missing log, wrong permissions (\u2260 0600), UTF-8 BOM present, CRLF line endings, empty file * line-level: invalid UTF-8 (with byte offset), embedded control characters, unparsable lines, timestamps out of order, duplicate `id:` and `aid:` (tracked separately so a stanza/archive id collision isn't double-reported) * cross-line: broken `corrects:` references whose target id is not present in the file - `/history backend` \u2014 show currently active backend - Active backend indicator `[sqlite]` / `[flatfile]` in the status bar next to the JID - Roster-JID autocomplete for verify / export / import - export and import open a SQLite handle on demand when the flatfile backend is currently active, so migration works regardless of which backend is live Tests - Unit: database_export (parser round-trip, escape/unescape, dedup key stability, JID normalisation), database_stress (14 cases exercising rapid writes, large messages, deep LMC chains, MAM dedup, concurrent contacts) - Functional: history persistence across reconnects, export / import round-trip with content equality, MUC migration, timestamp normalisation across timezones - Bench harness P1\u2013P5 (synthetic load: bulk insert, time-range read, page-up scroll, MAM ingest, mixed workload) and failure modes F1\u2013F17 (page-up cursor and forward-iteration symmetry, oversized lines, MAM dedup, LMC depth and cycles, BOM/CRLF, missing log, empty file, mtime+inode flip, broken corrects, etc.) - All bench tests integrate with the existing make targets and emit CSV rows for baseline comparison Author: jabber.developer2 <jabber.developer2@jabber.space> Reviewed-by: jabber.developer <jabber.developer@jabber.space>	2026-05-05 19:26:07 +00:00