Add backend-agnostic functional tests for database message persistence.
All tests work with both SQLite and flat-file backends.
New tests in test_history.c:
- message_db_history_on_reopen: basic incoming write+read round-trip
- message_db_history_multiple: 3 messages from different resources
- message_db_history_contact_isolation: buddy1 msg absent from buddy2
- message_db_history_special_chars: XML entities survive decode+DB
- message_db_history_outgoing: sent message persists across reopen
- message_db_history_dialog: outgoing + incoming both in history
- message_db_history_empty: no crash on contact with no history
- message_db_history_long_message: 1000+ char body not truncated
- message_db_history_newline: embedded LF stored correctly
- message_db_history_service_chars: backslash pipe percent braces etc
- message_db_history_verify: /history verify (ifdef HAVE_HISTORY_VERIFY)
- message_db_history_lmc: XEP-0308 correction replaces original in DB
Rebalance test groups for parallel execution (19/22/22/21):
- Group 1: Connect, Ping, Rooms, Software, LastActivity
- Group 2: Message, Receipts, Roster, DB History
- Group 3: Chat Session, Presence, Disconnect
- Group 4: MUC, Carbons
DRAFT — not yet tested end-to-end with a live XMPP session.
New commands:
/history export [<jid>] — copy messages from SQLite to flat-file
/history import [<jid>] — copy messages from flat-file to SQLite
Both merge with existing data; duplicates are skipped using stanza-id
or a SHA-256 fallback key (timestamp + sender + body prefix).
Implementation (src/database_export.c):
- Export: paginate SQLite via get_previous_chat, read existing flatfile
for dedup, write merged result via ff_write_line + atomic rename
- Import: parse flatfile lines via ff_parse_line, build dedup set from
SQLite, insert new messages via add_incoming (preserves original
timestamps for both directions)
- List all contacts: db_sqlite_list_contacts() queries UNION of
DISTINCT from_jid/to_jid; flatfile enumerates flatlog directories
Wiring:
- database.h: declare export/import functions + db_sqlite_list_contacts
- database_sqlite.c: add db_sqlite_list_contacts()
- cmd_defs.c: add export/import to /history synopsis and args
- cmd_funcs.c: add export/import handlers in cmd_history()
- cmd_ac.c: add 'export' and 'import' to history_ac
- Makefile.am: add database_export.c to core_sources
- stub_database.c: add stubs for test linking
- profanity.1: document export/import in man page
All code guarded with #ifdef HAVE_SQLITE — builds cleanly without it.
Build system:
- Add --without-sqlite configure flag (AM_CONDITIONAL BUILD_SQLITE)
- Guard database_sqlite.c with HAVE_SQLITE in Makefile.am, database.c,
database.h and stub_database.c
- Fall back to flatfile when SQLite not compiled in
Security (database_flatfile.c):
- MAM dedup: skip incoming messages with duplicate stanza-id (archive_id)
- LMC sender validation: reject corrections from mismatched JIDs
- Add flock() advisory locking to prevent interleaved writes from
concurrent instances
- Check fprintf return when writing file header
Autocomplete (cmd_ac.c):
- Add 'flatfile' to /privacy logging autocomplete
- Add dedicated /history autocomplete with on/off/verify (was boolean-only)
Code quality:
- Fix fwrite return type: size_t not ssize_t (database_flatfile_parser.c)
- Fix mixed allocators in ff_readline: use malloc() consistently for
overlength-line fallback instead of g_strdup()
- Fix index alignment in _ff_state_extend: use local counter so index
step doesn't depend on total_lines from initial build
Documentation:
- Fix page: correct directory structure (history.log not per-day files)
- Fix page: add aid:{archive_id} to line format example
- Fix page: missing newline before .SH BUGS
Add pluggable storage backend abstraction (vtable) to the database layer,
allowing selection between SQLite (default) and a new flat-file backend
that stores messages as human-readable plain text files.
New files:
- database_sqlite.c: extracted SQLite backend from database.c
- database_flatfile.c: plain text backend with tolerant parser,
LMC correction chains, UTF-8/BOM/CRLF handling, integrity checks
Commands:
- /privacy logging flatfile — switch to flat-file backend
- /history verify [<jid>] — check integrity of stored history
Fixes:
- Add missing 'off' guard in flatfile backend
- Enable CHLOG/HISTORY prefs when switching to flatfile mode
Logs stored in ~/.local/share/profanity/flatlog/{account}/{contact}/{date}.log
Format: {ISO8601} [{type}|{enc}|id:{id}|corrects:{id}] {sender}: {message}
Updated: CHANGELOG, CONTRIBUTING.md, profrc.example, man page, cmd_defs,
Makefile.am, test stubs, functional test support (PROF_FLATFILE=1)
- Add 8 tests for disco info and disco items commands
- Fix XEP-0030 compliance bug: show message for empty disco#items results
- Tests cover: identity display, features, server/jid queries, error handling,
items display, empty results, and connection requirement
Major changes:
Run 4 build configurations in parallel with Valgrind on Linux
Add test failure detection verification (meta-test)
Port allocation per build to prevent conflicts in parallel runs
Add --coverage-only flag for dedicated coverage builds
Code quality:
Add TEST_GROUPS constant, CMOCKA patterns, helper functions
Organize ci-build.sh into sections
Split functional tests into 4 parallel groups and add check-functional-parallel target (~3x faster CI runs).
Add branch-aware LCOV coverage reporting with new --enable-coverage option and lcov summary in CI pipeline.
Enable ccache via -C configure flag for faster recompilations.
Install lcov in all Docker images and use --depth 1 git clones + parallel make -j$(nproc) for quicker container builds.
Update CONTRIBUTING.md with instructions for parallel test groups and adding new ones.
All changes are tightly related CI/performance improvements developed in sequence. No external service uploads (e.g. Codecov skipped due to Gitea incompatibility).
As 9f2abc75 accidentally got the ordering of some of the includes wrong,
I decided to propose my initial solution again.
Additional to that, I've opened a MR against CMocka to solve this on
their side, since I believe that the current way this is done is not
sustainable [0].
[0] https://gitlab.com/cmocka/cmocka/-/merge_requests/91
Fixes: 9f2abc75 ("Fix tests with gcc15 (uintptr_t)")
Signed-off-by: Steffen Jaeckel <s@jaeckel.eu>
CProof note: our new tests need to also be updated.
* Also pass `$*` to `configure` when invoking `ci-build.sh`, so one can
e.g. run `./ci-build.sh --without-xscreensaver`
Signed-off-by: Steffen Jaeckel <s@jaeckel.eu>
While trying to get the unit tests working again I stumbled over all those
things that I thought could be better^TM.
Now we also know "TODO: why does this make the test fail?" - because
the unit tests are brittle AF ... and we have to init the subsystems
we use in the test, otherwise the cleanup will fail...
BTW. you can now also only run a single test ... or a pattern or so ...
you'd have to read how `cmocka_set_test_filter()` works exactly.
Signed-off-by: Steffen Jaeckel <s@jaeckel.eu>
Revert "Merge pull request #1943 from H3rnand3zzz/gmainloop
This reverts commit 609fde0998, reversing
changes made to 2ec94064ed.
Revert "Merge pull request #1948 from H3rnand3zzz/fix/rl-less-refreshes"
This reverts commit 11762fd2b0, reversing
changes made to 609fde0998.
We have got several issues, that we don't quite see how to solve, with
the merge of the gmainloop PR.
* Slashguard is broken (#1955) (though #1956 could fix that)
* One person reported problems with copy paste selection via mouse
* Some input buffer seems not to be cleared correctly
It happened that I was debugging profanity used `/connect` and typed
the password. I then debugged so long that a time out occurred, so
profanity disconnected. Then it printed "unknown command: $password".
There was something else that I forgot now.
Bottomline is: so far we didn't get it right so we will undo these
changes until someone proposes a working solution.
We got a slight performance increase (apparently noticable when
alt+mouse scrolling) but got too many issues with this change.
Only nicknames, photos, birthdays, addresses, telephone numbers, emails,
JIDs, titles, roles, notes, and URLs are supported
Due to the synopsis array not having enough space, `/vcard photo
open-self` and `/vcard photo save-self` are not documented properly in
the synopsis section of the `/vcard` command, but they are documented in
the arguments section
Fixed memory leak in vcard autocomplete (thanks to debXwoody)
The profanity-internal mechanism to allow connecting to a server isn't
easily portable to cURL. Therefor introduce a profanity-specific CAfile
which is managed individually and will be configured in libcurl calls.
Signed-off-by: Steffen Jaeckel <jaeckel-floss@eyet-services.de>
src/pgp/gpg.c:p_ox_gpg_readkey
Used to read a public key from a file. The function will return the fingerprint
of the file and the base64 encoded key.
src/xmpp/ox.[hc]
ox_announce_public_key(const char* const filename) can be called from the /ox
announce <filename> command. The key within the file will be pushed on PEP and
the Metadata node will be set.
Issue: #1331
I plan to save all messages in an SQLite db.
For retrieving information it's nicer than having it in a text file.
We will have more info in there and easier to parse it.
This will also be good for later MAM
(https://github.com/profanity-im/profanity/issues/660).
Regular text files will still be an option for users so that they can
easily grep them and do whatever they like.
Internally Profanity will only use the SQLite db.
