feat: complete field parity, harden export/import, add tests

Export/Import improvements:
- Replace pagination with direct SQL query (db_sqlite_get_all_chat)
- Wrap import in SQL transaction with rollback on error
- Add fsync before fclose in export for data safety
- Sort merged output by timestamp with secondary key (stanza_id, from_jid)
- Export archive_id and marked_read from SQLite (lossless migration)
- Add progress indication every 500 messages during write/import
- Expand dedup key body prefix from 64 to 256 chars
- Fix g_slist_append O(n²) → g_slist_prepend + g_slist_reverse O(n)

Field parity (to_jid, to_resource, marked_read):
- Add fields to ff_parsed_line_t struct
- Write/parse to:|to_res:|read: metadata tags in flatfile format
- Pass to_resource through _ff_add_message and all callers
- Add marked_read to ProfMessage struct with -1 default (unset)
- Preserve fields across export/import round-trips

Tests (19 new: 11 unit + 8 functional):
- Unit: to_jid_and_marked_read, bracket_in_stanza_id, backslash_in_resource,
  mucpm_type, all_enc_types, crlf_handling, to_jid_special_chars,
  multiple_lines, parsed_line_free_null_safe, no_space_rejected,
  unclosed_bracket
- Functional: export_idempotent_no_duplicates, export_lmc_correction_survives,
  switch_preserves_old_backend_data, export_all_contacts,
  import_double_dedup, verify_after_export,
  switch_backends_independent_messages, export_empty_contact
- Rebalance test groups: move Chat Session from Group 3 to Group 4
  (25/33/30/27 instead of 25/33/36/21)
- Remove hardcoded test counts from group comments

Man page:
- Document /history switch sqlite|flatfile
This commit is contained in:
2026-03-09 12:14:41 +03:00
parent d5d67592ab
commit e91e8403b0
16 changed files with 1037 additions and 146 deletions

View File

@@ -320,8 +320,8 @@ _ff_get_state(const char* contact_barejid)
static void
_ff_add_message(const char* type, const char* stanza_id, const char* archive_id,
const char* replace_id, const char* from_barejid, const char* from_resource,
const char* to_barejid, const char* message_text, GDateTime* timestamp,
prof_enc_t enc)
const char* to_barejid, const char* to_resource, const char* message_text,
GDateTime* timestamp, prof_enc_t enc)
{
auto_gchar gchar* pref_dblog = prefs_get_string(PREF_DBLOG);
@@ -408,7 +408,9 @@ _ff_add_message(const char* type, const char* stanza_id, const char* archive_id,
ff_write_line(fp, date_fmt, type, ff_get_message_enc_str(enc),
stanza_id, archive_id, replace_id,
from_barejid, from_resource, effective_msg);
from_barejid, from_resource,
to_barejid, to_resource, -1,
effective_msg);
fflush(fp);
// fclose also releases the flock
@@ -586,7 +588,7 @@ _flatfile_add_incoming(ProfMessage* message)
_ff_add_message(type, message->id, message->stanzaid, message->replace_id,
message->from_jid->barejid, message->from_jid->resourcepart,
to_jid->barejid, message->plain,
to_jid->barejid, to_jid->resourcepart, message->plain,
message->timestamp, message->enc);
}
@@ -597,7 +599,7 @@ _flatfile_add_outgoing_chat(const char* const id, const char* const barejid,
const Jid* myjid = connection_get_jid();
_ff_add_message("chat", id, NULL, replace_id,
myjid ? myjid->barejid : "me", myjid ? myjid->resourcepart : NULL,
barejid, message, NULL, enc);
barejid, NULL, message, NULL, enc);
}
static void
@@ -607,7 +609,7 @@ _flatfile_add_outgoing_muc(const char* const id, const char* const barejid,
const Jid* myjid = connection_get_jid();
_ff_add_message("muc", id, NULL, replace_id,
myjid ? myjid->barejid : "me", myjid ? myjid->resourcepart : NULL,
barejid, message, NULL, enc);
barejid, NULL, message, NULL, enc);
}
static void
@@ -617,7 +619,7 @@ _flatfile_add_outgoing_muc_pm(const char* const id, const char* const barejid,
const Jid* myjid = connection_get_jid();
_ff_add_message("mucpm", id, NULL, replace_id,
myjid ? myjid->barejid : "me", myjid ? myjid->resourcepart : NULL,
barejid, message, NULL, enc);
barejid, NULL, message, NULL, enc);
}
// =========================================================================