- OTR: strip the whitespace tag by shifting the full message tail incl. the NUL, not tag_length bytes, so the body is no longer duplicated for messages longer than the tag.
- presence: snapshot the resource fields before connection_add_available_resource() takes ownership, removing a use-after-free in the own-presence path.
- OMEMO: propagate _omemo_finalize_identity_load() failure on connect (log + cons_show_error + stop) instead of leaving OMEMO silently unavailable.
- OMEMO: guard NULL fingerprint decode in _omemo_fingerprint_decode / omemo_is_trusted_identity / omemo_trust and log every decode failure instead of failing silently.
- stanza-id: gate XEP-0359 dedup on disco urn:xmpp:sid:0 (the `by` JID or its domain), falling back to no-dedup when caps are unknown; add functional tests for trusted vs untrusted server.
- accounts: narrow the group-name sanitizer to the characters GKeyFile forbids in headers ([ ] \n \r), keeping `=` and `#`, so read/write stays symmetric.
- build: re-introduce compiler/sanitizer flags in a Pikaur-safe form (opt-in sanitizers, -Wsign-compare) and fix the resulting -Wsign-compare warnings (incl. proftest _mkdir_recursive).fix: OTR/presence/OMEMO correctness, stanza-id disco gate, build hardening
Author: jabber.developer2 <jabber.developer2@jabber.space>
A flat-file alternative to the SQLite chatlog backend with runtime
switching, full migration tooling, integrity verification, and a
synthetic load harness. SQLite remains the default; both backends share
one dispatch layer (db_backend_t vtable) so callers don't change.
Storage layout
- Per-contact append-only `flatlog/<account>/<contact>/history.log`
under XDG_DATA_HOME, one line per message
- Single-line file header with embedded format-version marker
(FLATFILE_FORMAT_VERSION); reader warns on missing or mismatched
marker, writer and checker stay in sync via preprocessor
stringification
- Deterministic key=value metadata (`id`, `aid`, `corrects`, `to`,
`to_res`, `read`) plus escaped body \u2014 `\|`, `\]`, `\\`, `\n`, `\r`
literals prevent log injection
- Sparse byte-offset index (FF_INDEX_STEP=500) per contact for
O(log n) time-range lookups; rebuilt on inode / size / mtime
change, extended in-place when the file just grew
- Per-contact GHashTable caches for archive_id presence and
stanza_id \u2192 from_jid mapping (O(1) MAM dedup, O(1) LMC sender
validation)
Hardening
- Path-traversal protection: JID directory name normalisation
(`@` \u2192 `_at_`, slashes and `..` rejected at construction); every
per-contact path is anchored under the account's flatlog/
directory and validated before open
- Symlink-attack protection: every fopen / open uses O_NOFOLLOW; on
ELOOP the operation aborts with an error rather than following
- Filesystem permissions: log files created with mode 0600,
directories with mode 0700; both enforced at creation, verified
on each open and reported on drift by `/history verify`
- Atomic crash-safe export: write to a temp file via mkstemp (mode
0600, random suffix, no name collisions between concurrent
exports), fsync, then rename \u2014 partial state never replaces the
live file
- Concurrency: advisory flock(LOCK_EX) held for the duration of
every write, including append from live messages and full rewrite
from export, so two profanity processes can't interleave bytes
on the same log
- DoS / abuse guards:
* FF_MAX_LINE_LEN = 10 MB \u2014 lines longer than this are rejected
at read with a warning; the parser will not allocate
unbounded memory for a single record
* FF_MAX_LMC_DEPTH = 100 \u2014 `corrects:` chain walk stops at this
depth and emits a warning, preventing a malicious correction
cycle from spinning the apply pass
* FF_VERSION_SCAN_MAX = 16 \u2014 header version probe never reads
past 16 leading comment lines, even on garbage input
* Empty / inverted byte-range early-return in page-up read path
so a malformed time filter cannot cause an unbounded scan
* Zero-entry index guard so a file whose every line failed to
parse cannot cause a NULL deref on later page-up
- LMC sender validation: an incoming correction whose sender does
not match the original message's sender is rejected at write
time and surfaced via cons_show_error; a cycle in the apply pass
is broken via a visited-set
- jid_create_from_bare_and_resource treats NULL, empty string, and
the literal "(null)" as no resource and returns a bare jid;
similar normalisation for barejid eliminates the legacy
"user@host/(null)" artefact that leaked into stored fulljids
whenever g_strdup_printf("%s", NULL) ran inside create_fulljid
Commands
- `/history switch sqlite|flatfile` \u2014 runtime backend swap, closes
the old backend and opens the new one without reconnecting
- `/history export [<jid>]` \u2014 SQLite -> flat-file, merging with any
existing flatlog (dedup keyed on a SHA-256 hash mixing stanza_id,
timestamp, from_jid, body \u2014 robust against id reuse by older
clients)
- `/history import [<jid>]` \u2014 flat-file -> SQLite, same merge
semantics, runs inside a single SQLite transaction with rollback
on per-contact failure
- `/history verify [<jid>]` \u2014 integrity check; emits a structured
list of issues (ERROR / WARNING / INFO) per file:
* file-level: missing log, wrong permissions (\u2260 0600), UTF-8
BOM present, CRLF line endings, empty file
* line-level: invalid UTF-8 (with byte offset), embedded
control characters, unparsable lines, timestamps out of
order, duplicate `id:` and `aid:` (tracked separately so a
stanza/archive id collision isn't double-reported)
* cross-line: broken `corrects:` references whose target id is
not present in the file
- `/history backend` \u2014 show currently active backend
- Active backend indicator `[sqlite]` / `[flatfile]` in the status
bar next to the JID
- Roster-JID autocomplete for verify / export / import
- export and import open a SQLite handle on demand when the
flatfile backend is currently active, so migration works
regardless of which backend is live
Tests
- Unit: database_export (parser round-trip, escape/unescape, dedup
key stability, JID normalisation), database_stress (14 cases
exercising rapid writes, large messages, deep LMC chains, MAM
dedup, concurrent contacts)
- Functional: history persistence across reconnects, export /
import round-trip with content equality, MUC migration,
timestamp normalisation across timezones
- Bench harness P1\u2013P5 (synthetic load: bulk insert, time-range
read, page-up scroll, MAM ingest, mixed workload) and failure
modes F1\u2013F17 (page-up cursor and forward-iteration symmetry,
oversized lines, MAM dedup, LMC depth and cycles, BOM/CRLF,
missing log, empty file, mtime+inode flip, broken corrects, etc.)
- All bench tests integrate with the existing make targets and
emit CSV rows for baseline comparison
Author: jabber.developer2 <jabber.developer2@jabber.space>
Reviewed-by: jabber.developer <jabber.developer@jabber.space>
Split functional tests into 4 parallel groups and add check-functional-parallel target (~3x faster CI runs).
Add branch-aware LCOV coverage reporting with new --enable-coverage option and lcov summary in CI pipeline.
Enable ccache via -C configure flag for faster recompilations.
Install lcov in all Docker images and use --depth 1 git clones + parallel make -j$(nproc) for quicker container builds.
Update CONTRIBUTING.md with instructions for parallel test groups and adding new ones.
All changes are tightly related CI/performance improvements developed in sequence. No external service uploads (e.g. Codecov skipped due to Gitea incompatibility).
* Also pass `$*` to `configure` when invoking `ci-build.sh`, so one can
e.g. run `./ci-build.sh --without-xscreensaver`
Signed-off-by: Steffen Jaeckel <s@jaeckel.eu>
This commit fixes few issues related to OMEMO autodetection:
1. Absence of libsignal-protocol-c doesn't turn OMEMO off, just prints a
notice message. It should (a) set BUILD_OMEMO=false and (b) terminate
with an error on --enable-omemo=yes.
2. Check for gcrypt continues even if libsignal-protocol-c fails. In this
case, LIBS variable can be updated with unneeded -lgcrypt.
3. Similarly to item 2., if libsignal-protocol-c is present, but gcrypt
isn't, variable LIBS is updated with unneeded library.
To resolve the above issues, use intermediate variable OMEMO_LIBS to
accumulate required libraries and set BUILD_OMEMO only when all checks
are passed.
In 28a9605a1 we migrated from AC_CHECK_LIB which defines HAVE_LIBXSS automatically. With pkg-config way you need it explicit. And also x11 is needed or else linking will fail missing XFree().
Patch provided by Paul Fertser and comitted by jubalh.
Thanks Paul!
Fix https://github.com/profanity-im/profanity/issues/1695
Before we got an error when libqrencode was not installed:
`src/ui/console.c:52:10: fatal error: qrencode.h: No such file or
directory`
Which looked like HAVE_QRENCODE was true.
config.status showed:
`config.status:D["HAVE_QRENCODE"]=" 0"`
Holger pointed to me that there is not just true and false but defined
and undefined.
So one solution was to use `#if HAVE_QRENCODE == 1` to check for the
actual value.
Or dont define HAVE_QRENCODE in the non present case at all.
In all the other HAVE_ variables we use this approach.
I think i at first chose the wrong way out of confusion with BUILD_ and
HAVE_.
Option -ggdb3 (same as -g3) doesn't affect code-generation, just adds
to the binary size of an executable. Automake strips installed
binaries automatically when "make install-strip" is run so enabling
this option permanently doesn't affect the end result while allowing
for hassle-free interactive debugging.
If one is running a stripped binary it's still possible to attach GDB
by process PID and then use "file /path/to/unstripped/profanity" to
get all symbols and lines data.
Autoconf can pre-populate this variable with essential parameters so
it should be appended to rather than overridden.
While at it, don't miss to append CFLAGS for libstrope which is needed
if it's installed to a non-default location.
AM_INIT_AUTOMAKE requires AC_ARG_PROGRAM so should go after
AC_CANONICAL_TARGET for proper program name mangling.
This fixes "AC_ARG_PROGRAM was called before AC_CANONICAL_TARGET"
warning as emitted by autoconf 2.69.
Fixing build failure detected by sr.ht for OpenBSD:
```
src/ui/inputwin.c: In function '_inp_rl_startup_hook':
src/ui/inputwin.c:444:5: error: implicit declaration of function 'rl_bind_keyseq'; did you mean 'rl_bind_key'? [-Werror=implicit-function-declaration]
444 | rl_bind_keyseq("\\e1", _inp_rl_win_1_handler);
| ^~~~~~~~~~~~~~
| rl_bind_key
cc1: all warnings being treated as errors
```
Seems like both OSX and OpenBSD need CPPFLAGS here.
Previously it relied on AX_PYTHON_DEVEL, which in turn executes
python-config to get the build flags. However this does not work while
cross compiling because we can't execute the python-config build for the
target platform. To circumvent this problem the python build flags are
now queried via pkgconfig, which has the drawback of not having some
extra build flags, but they do not seem to be needed.
I tested this patch with the termux build system and it build without
their existing hack of injecting python after the configure step. I also
tested non cross compile build on Arch Linux and it also still works.
Fixes#851
Remove support for libsignal-protocol-c < 2.3.2.
Debian 10 uses 2.3.2, Debian 11 and 12 use 2.3.3.
openSUSE from 15.2 onward uses 2.3.3.
Fedora since 28 uses 2.3.2.
We should be good.
