Add backend-agnostic functional tests for database message persistence.
All tests work with both SQLite and flat-file backends.
New tests in test_history.c:
- message_db_history_on_reopen: basic incoming write+read round-trip
- message_db_history_multiple: 3 messages from different resources
- message_db_history_contact_isolation: buddy1 msg absent from buddy2
- message_db_history_special_chars: XML entities survive decode+DB
- message_db_history_outgoing: sent message persists across reopen
- message_db_history_dialog: outgoing + incoming both in history
- message_db_history_empty: no crash on contact with no history
- message_db_history_long_message: 1000+ char body not truncated
- message_db_history_newline: embedded LF stored correctly
- message_db_history_service_chars: backslash pipe percent braces etc
- message_db_history_verify: /history verify (ifdef HAVE_HISTORY_VERIFY)
- message_db_history_lmc: XEP-0308 correction replaces original in DB
Rebalance test groups for parallel execution (19/22/22/21):
- Group 1: Connect, Ping, Rooms, Software, LastActivity
- Group 2: Message, Receipts, Roster, DB History
- Group 3: Chat Session, Presence, Disconnect
- Group 4: MUC, Carbons
Add log_database_switch_backend() that closes the current backend,
updates PREF_DBLOG preference, and reinitializes with the new backend
without requiring a reconnect or restart.
New command: /history switch sqlite|flatfile
- Validates connection state and backend name
- Skips if already using the requested backend
- Autocomplete support for the subcommand
DRAFT — not yet tested end-to-end with a live XMPP session.
New commands:
/history export [<jid>] — copy messages from SQLite to flat-file
/history import [<jid>] — copy messages from flat-file to SQLite
Both merge with existing data; duplicates are skipped using stanza-id
or a SHA-256 fallback key (timestamp + sender + body prefix).
Implementation (src/database_export.c):
- Export: paginate SQLite via get_previous_chat, read existing flatfile
for dedup, write merged result via ff_write_line + atomic rename
- Import: parse flatfile lines via ff_parse_line, build dedup set from
SQLite, insert new messages via add_incoming (preserves original
timestamps for both directions)
- List all contacts: db_sqlite_list_contacts() queries UNION of
DISTINCT from_jid/to_jid; flatfile enumerates flatlog directories
Wiring:
- database.h: declare export/import functions + db_sqlite_list_contacts
- database_sqlite.c: add db_sqlite_list_contacts()
- cmd_defs.c: add export/import to /history synopsis and args
- cmd_funcs.c: add export/import handlers in cmd_history()
- cmd_ac.c: add 'export' and 'import' to history_ac
- Makefile.am: add database_export.c to core_sources
- stub_database.c: add stubs for test linking
- profanity.1: document export/import in man page
All code guarded with #ifdef HAVE_SQLITE — builds cleanly without it.
Build system:
- Add --without-sqlite configure flag (AM_CONDITIONAL BUILD_SQLITE)
- Guard database_sqlite.c with HAVE_SQLITE in Makefile.am, database.c,
database.h and stub_database.c
- Fall back to flatfile when SQLite not compiled in
Security (database_flatfile.c):
- MAM dedup: skip incoming messages with duplicate stanza-id (archive_id)
- LMC sender validation: reject corrections from mismatched JIDs
- Add flock() advisory locking to prevent interleaved writes from
concurrent instances
- Check fprintf return when writing file header
Autocomplete (cmd_ac.c):
- Add 'flatfile' to /privacy logging autocomplete
- Add dedicated /history autocomplete with on/off/verify (was boolean-only)
Code quality:
- Fix fwrite return type: size_t not ssize_t (database_flatfile_parser.c)
- Fix mixed allocators in ff_readline: use malloc() consistently for
overlength-line fallback instead of g_strdup()
- Fix index alignment in _ff_state_extend: use local counter so index
step doesn't depend on total_lines from initial build
Documentation:
- Fix page: correct directory structure (history.log not per-day files)
- Fix page: add aid:{archive_id} to line format example
- Fix page: missing newline before .SH BUGS
Replace per-contact directory structure with a single flat file per contact.
Each file uses pipe-delimited records with a sparse byte-offset index
that is rebuilt on demand and cached in memory.
- Rewrite _ff_write_record / _ff_read_records for single-file format
- Add sparse index (every Nth record) for O(log N) seek on history read
- Cursor-based pagination: _ff_get_previous_chat uses saved byte offset
- LMC corrections applied in-place during read (_ff_apply_lmc)
- Newline escaping (\n) in message bodies for line-based storage
- Simplify verify: parse + timestamp order + LMC reference check
- Update parser helpers for new field layout
Add pluggable storage backend abstraction (vtable) to the database layer,
allowing selection between SQLite (default) and a new flat-file backend
that stores messages as human-readable plain text files.
New files:
- database_sqlite.c: extracted SQLite backend from database.c
- database_flatfile.c: plain text backend with tolerant parser,
LMC correction chains, UTF-8/BOM/CRLF handling, integrity checks
Commands:
- /privacy logging flatfile — switch to flat-file backend
- /history verify [<jid>] — check integrity of stored history
Fixes:
- Add missing 'off' guard in flatfile backend
- Enable CHLOG/HISTORY prefs when switching to flatfile mode
Logs stored in ~/.local/share/profanity/flatlog/{account}/{contact}/{date}.log
Format: {ISO8601} [{type}|{enc}|id:{id}|corrects:{id}] {sender}: {message}
Updated: CHANGELOG, CONTRIBUTING.md, profrc.example, man page, cmd_defs,
Makefile.am, test stubs, functional test support (PROF_FLATFILE=1)
Replace 16 interactive UI setup commands with pre-written profrc file
containing [ui] and [notifications] sections (~1800ms saved per test).
Replace sleep(1) + blocking waitpid with close(pty fd) → SIGHUP →
polling waitpid(WNOHANG) → SIGTERM/SIGKILL fallback chain (~4900ms
saved per test).
Remove post-stbbr_start() and post-stbbr_stop() sleeps — bind+listen
completes synchronously before stbbr_start() returns, and
pthread_join() in stbbr_stop() guarantees socket cleanup (~200ms saved).
Add PORTS_PER_GROUP=50 isolated port ranges per test group to enable
safe parallel execution of 4 groups without port conflicts.
Add 7 new tests for /disco command:
- disco_items_to_jid: query items to specific JID
- disco_info_empty_result: handle empty disco#info response
- disco_info_multiple_identities: multiple identity elements
- disco_info_without_name: identity without optional name attr
- disco_items_without_name: items without optional name attr
- disco_info_service_unavailable: error handling for info
- disco_items_error_handling: error handling for items (XEP-0030 §7)
The disco_items_error_handling test documents a bug where disco#items
errors are silently ignored (unlike disco#info which handles them).
This violates XEP-0030 Section 7 which requires error feedback to user.
- Add 8 tests for disco info and disco items commands
- Fix XEP-0030 compliance bug: show message for empty disco#items results
- Tests cover: identity display, features, server/jid queries, error handling,
items display, empty results, and connection requirement
Per XEP-0030 section 3.1: 'if an entity has no associated items,
it MUST return an empty <query/> element.'
The client should display 'No service discovery items for X' when
receiving an empty result, not silently ignore it.
Previous version introduced in commit f28655c5c (2016-05-08) which added an early
return when child == NULL, preventing the message from being shown.
Major changes:
Run 4 build configurations in parallel with Valgrind on Linux
Add test failure detection verification (meta-test)
Port allocation per build to prevent conflicts in parallel runs
Add --coverage-only flag for dedicated coverage builds
Code quality:
Add TEST_GROUPS constant, CMOCKA patterns, helper functions
Organize ci-build.sh into sections
Remove -c http.sslverify=false from all git clones
(enables proper TLS verification, closes MITM risk).
Explicitly install ca-certificates in every CI Docker image.
Split functional tests into 4 parallel groups and add check-functional-parallel target (~3x faster CI runs).
Add branch-aware LCOV coverage reporting with new --enable-coverage option and lcov summary in CI pipeline.
Enable ccache via -C configure flag for faster recompilations.
Install lcov in all Docker images and use --depth 1 git clones + parallel make -j$(nproc) for quicker container builds.
Update CONTRIBUTING.md with instructions for parallel test groups and adding new ones.
All changes are tightly related CI/performance improvements developed in sequence. No external service uploads (e.g. Codecov skipped due to Gitea incompatibility).
- Add prof.supp with pthread TLS suppressions
- Update ci-build.sh with test configurations
- Document functional test best practices in CONTRIBUTING.md
- Replace stbbr_for_id() with stbbr_for_query()/stbbr_send()
- Content-based stubbing matches stanzas by namespace instead of ID
- Use regex assertions for flexible output matching
- Fix timing issues in chat_session and presence tests
- Add TERM=xterm-256color for PTY support in functional tests
- Disable SSL verification for git clone in restricted networks
- Add error handling for Arch reflector installation
The /connect command can take: account + server <s> + port <p> + tls <t> + auth <a>
which totals 9 arguments, not 7. This fixes argument parsing for full command usage.
- Track unread message indicators and paging state in the window component
- Clear and restore unread markers correctly when navigating between conversations
- Suppress buffer updates and message printing while viewing history to prevent
unwanted scroll jumps and viewport lock
Also includes minor code-style, whitespace, and comment cleanups as well as
updated documentation for metrics and buffer handling.
Add debug output tracking connection lifecycle.
- Track disconnects and reconnects
- Record session login, logout, and reconnection attempts
- Use [CONNDBG] tag for easy log filtering
Add prof_get_current_window API to retrieve the title of the currently active window, matching the titlebar display.
The feature is especially useful to track currently used plugin window.
Remove autodisable logic (prefs_set_autoping(0)) and early return in autoping
to treat intermittent false negatives from connection_supports(XMPP_FEATURE_PING).
Add one-time error display with debug features print for monitoring; warn on first failing
check without aborting to maintain functionality while investigating root cause.
Prior commit (6ad8a190) did not properly handle overflow by long (9000 lines) message,
to address this issue, multiple changes were made:
- Add lines recalculation on win_redraw
- Move `prof_buff_t` struct to header file so its lines could be externally changed
- Call win_redraw once overflow is detected: it allows to recalculate sizes in lines and apply changes to the buffer
Refactor buffer trimming logic to use dynamic line counts (_lines per entry)
against PAD_SIZE (10k) instead of fixed entry count (MAX_BUFFER_SIZE=200). This
prevents premature cleanups for short messages, reduces scrolling disruptions
during history loads, and scales better with rendered content size.
Delete oldest/newest entry from opposite end only when total lines approach
ncurses pad limit, minimizing unnecessary deletions.
Simplify overflow warning log by removing unused MAX_BUFFER_SIZE reference.
Update buffer.c header for improved DX:
- Add detailed module overview explaining role (in-memory UI buffer vs. DB
persistence), key features (trimming, metadata), and integration (ncurses).
- Shorten copyright/license boilerplate to concise pointer (LICENSE ref).
- Add fork notice; preserve original copyrights per GPLv3.
Partially addresses #36 (stuck scroll mitigation via fewer cleanups)
Related to #36
- Defined callback protocols for register_command, register_timed, win_create.
- Fixed Sphinx callable warnings with proper type hints.
- Align docs with the code
