Add log_database_switch_backend() that closes the current backend,
updates PREF_DBLOG preference, and reinitializes with the new backend
without requiring a reconnect or restart.
New command: /history switch sqlite|flatfile
- Validates connection state and backend name
- Skips if already using the requested backend
- Autocomplete support for the subcommand
DRAFT — not yet tested end-to-end with a live XMPP session.
New commands:
/history export [<jid>] — copy messages from SQLite to flat-file
/history import [<jid>] — copy messages from flat-file to SQLite
Both merge with existing data; duplicates are skipped using stanza-id
or a SHA-256 fallback key (timestamp + sender + body prefix).
Implementation (src/database_export.c):
- Export: paginate SQLite via get_previous_chat, read existing flatfile
for dedup, write merged result via ff_write_line + atomic rename
- Import: parse flatfile lines via ff_parse_line, build dedup set from
SQLite, insert new messages via add_incoming (preserves original
timestamps for both directions)
- List all contacts: db_sqlite_list_contacts() queries UNION of
DISTINCT from_jid/to_jid; flatfile enumerates flatlog directories
Wiring:
- database.h: declare export/import functions + db_sqlite_list_contacts
- database_sqlite.c: add db_sqlite_list_contacts()
- cmd_defs.c: add export/import to /history synopsis and args
- cmd_funcs.c: add export/import handlers in cmd_history()
- cmd_ac.c: add 'export' and 'import' to history_ac
- Makefile.am: add database_export.c to core_sources
- stub_database.c: add stubs for test linking
- profanity.1: document export/import in man page
All code guarded with #ifdef HAVE_SQLITE — builds cleanly without it.
Build system:
- Add --without-sqlite configure flag (AM_CONDITIONAL BUILD_SQLITE)
- Guard database_sqlite.c with HAVE_SQLITE in Makefile.am, database.c,
database.h and stub_database.c
- Fall back to flatfile when SQLite not compiled in
Security (database_flatfile.c):
- MAM dedup: skip incoming messages with duplicate stanza-id (archive_id)
- LMC sender validation: reject corrections from mismatched JIDs
- Add flock() advisory locking to prevent interleaved writes from
concurrent instances
- Check fprintf return when writing file header
Autocomplete (cmd_ac.c):
- Add 'flatfile' to /privacy logging autocomplete
- Add dedicated /history autocomplete with on/off/verify (was boolean-only)
Code quality:
- Fix fwrite return type: size_t not ssize_t (database_flatfile_parser.c)
- Fix mixed allocators in ff_readline: use malloc() consistently for
overlength-line fallback instead of g_strdup()
- Fix index alignment in _ff_state_extend: use local counter so index
step doesn't depend on total_lines from initial build
Documentation:
- Fix page: correct directory structure (history.log not per-day files)
- Fix page: add aid:{archive_id} to line format example
- Fix page: missing newline before .SH BUGS
Add pluggable storage backend abstraction (vtable) to the database layer,
allowing selection between SQLite (default) and a new flat-file backend
that stores messages as human-readable plain text files.
New files:
- database_sqlite.c: extracted SQLite backend from database.c
- database_flatfile.c: plain text backend with tolerant parser,
LMC correction chains, UTF-8/BOM/CRLF handling, integrity checks
Commands:
- /privacy logging flatfile — switch to flat-file backend
- /history verify [<jid>] — check integrity of stored history
Fixes:
- Add missing 'off' guard in flatfile backend
- Enable CHLOG/HISTORY prefs when switching to flatfile mode
Logs stored in ~/.local/share/profanity/flatlog/{account}/{contact}/{date}.log
Format: {ISO8601} [{type}|{enc}|id:{id}|corrects:{id}] {sender}: {message}
Updated: CHANGELOG, CONTRIBUTING.md, profrc.example, man page, cmd_defs,
Makefile.am, test stubs, functional test support (PROF_FLATFILE=1)
- Introduce static helper `_truncate_datetime_suffix()` to safely trim datetime strings, removing unwanted suffixes like timezone offsets
- Replace manual string management with auto_gchar and g_strdup for safer, clearer ownership and to prevent leaks
- Add safety checks and logging warnings for unexpected datetime string lengths or null pointers
- Refactor _mam_rsm_id_handler to use the helper function and updated string handling
- Change log_database_get_previous_chat parameters for consistent ownership semantics, avoiding double frees and mem leaks
- Overall improve stability and prevent memory leaks during log database queries
This commit refines the existing logic in win_page_up() by:
- Improving comments to clearly explain the rationale behind adjusting
the scroll offset relative to the first buffer entry’s visual position,
helping future maintainers understand why this is necessary.
- Fixing offset recalculation to better handle cases where older messages
with variable heights are loaded from the DB, improving scroll smoothness.
- Changing the logging of negative *page_start values from warning to debug,
recognizing that this can be a normal scenario when insufficient history is loaded.
- Simplifying some conditionals and renaming variables for clearer intent.
No changes yet applied to win_page_down(), but similar improvements could
be considered in the future.
Overall, this enhances the robustness and user experience of scrolling up
in chat windows, while preserving existing functional logic.
Did this by waiting for a batch of MAM messages to arrive before
prepending them to the buffer. Also limited the number of messages
to fetch to 10 so that the user gets more frequent updates.
We now dont get the log files from the text files via chat_log_get_previous() anymore.
We use the sql backend via log_database_get_previous_chat().
So far it just has the same behaviour like chat_log_get_previous(),
except that in _chatwin_history() we don't pass the sender to
win_print_history() which should be fixed in a commit soon.
And log_database_get_previous_chat() can later easily be expanded to fix
https://github.com/profanity-im/profanity/issues/205.
I plan to save all messages in an SQLite db.
For retrieving information it's nicer than having it in a text file.
We will have more info in there and easier to parse it.
This will also be good for later MAM
(https://github.com/profanity-im/profanity/issues/660).
Regular text files will still be an option for users so that they can
easily grep them and do whatever they like.
Internally Profanity will only use the SQLite db.
