devs/cproof
1
2
Fork 1
Code Issues 43 Pull Requests 6 Actions Packages Projects 1 Releases Wiki Activity

feat: complete field parity, harden export/import, add tests

Browse Source 
Export/Import improvements:
- Replace pagination with direct SQL query (db_sqlite_get_all_chat)
- Wrap import in SQL transaction with rollback on error
- Add fsync before fclose in export for data safety
- Sort merged output by timestamp with secondary key (stanza_id, from_jid)
- Export archive_id and marked_read from SQLite (lossless migration)
- Add progress indication every 500 messages during write/import
- Expand dedup key body prefix from 64 to 256 chars
- Fix g_slist_append O(n²) → g_slist_prepend + g_slist_reverse O(n)

Field parity (to_jid, to_resource, marked_read):
- Add fields to ff_parsed_line_t struct
- Write/parse to:|to_res:|read: metadata tags in flatfile format
- Pass to_resource through _ff_add_message and all callers
- Add marked_read to ProfMessage struct with -1 default (unset)
- Preserve fields across export/import round-trips

Tests (19 new: 11 unit + 8 functional):
- Unit: to_jid_and_marked_read, bracket_in_stanza_id, backslash_in_resource,
  mucpm_type, all_enc_types, crlf_handling, to_jid_special_chars,
  multiple_lines, parsed_line_free_null_safe, no_space_rejected,
  unclosed_bracket
- Functional: export_idempotent_no_duplicates, export_lmc_correction_survives,
  switch_preserves_old_backend_data, export_all_contacts,
  import_double_dedup, verify_after_export,
  switch_backends_independent_messages, export_empty_contact
- Rebalance test groups: move Chat Session from Group 3 to Group 4
  (25/33/30/27 instead of 25/33/36/21)
- Remove hardcoded test counts from group comments

Man page:
- Document /history switch sqlite|flatfile
This commit is contained in:
jabber.developer2
2026-03-09 12:14:41 +03:00
parent 9f3020e40a
commit 507ef91b5f
16 changed files with 1037 additions and 146 deletions
Download Patch File Download Diff File Expand all files Collapse all files

29
src/database_flatfile_parser.c
View File

@@ -402,14 +402,16 @@ ff_readline(FILE* fp, gboolean* truncated)
void
ff_write_line(FILE* fp, const char* timestamp, const char* type, const char* enc,
const char* stanza_id, const char* archive_id, const char* replace_id,
const char* from_jid, const char* from_resource, const char* message_text)
const char* from_jid, const char* from_resource,
const char* to_jid, const char* to_resource, int marked_read,
const char* message_text)
{
// Escape metadata values from remote peers
auto_gchar gchar* safe_sid = ff_escape_meta_value(stanza_id);
auto_gchar gchar* safe_aid = ff_escape_meta_value(archive_id);
auto_gchar gchar* safe_rid = ff_escape_meta_value(replace_id);
// Build metadata section: [type|enc|id:...|aid:...|corrects:...]
// Build metadata section: [type|enc|id:...|aid:...|corrects:...|to:...|to_res:...|read:...]
GString* meta = g_string_new("[");
g_string_append(meta, type ? type : "chat");
g_string_append_c(meta, '|');
@@ -423,6 +425,17 @@ ff_write_line(FILE* fp, const char* timestamp, const char* type, const char* enc
if (safe_rid) {
g_string_append_printf(meta, "|corrects:%s", safe_rid);
}
if (to_jid && strlen(to_jid) > 0) {
auto_gchar gchar* safe_to = ff_escape_meta_value(to_jid);
g_string_append_printf(meta, "|to:%s", safe_to);
}
if (to_resource && strlen(to_resource) > 0) {
auto_gchar gchar* safe_tores = ff_escape_meta_value(to_resource);
g_string_append_printf(meta, "|to_res:%s", safe_tores);
}
if (marked_read >= 0) {
g_string_append_printf(meta, "|read:%d", marked_read ? 1 : 0);
}
g_string_append_c(meta, ']');
// Build sender — escape ": " in the resource part to prevent
@@ -573,6 +586,8 @@ ff_parsed_line_free(ff_parsed_line_t* pl)
g_free(pl->replace_id);
g_free(pl->from_jid);
g_free(pl->from_resource);
g_free(pl->to_jid);
g_free(pl->to_resource);
g_free(pl->message);
g_free(pl);
}
@@ -619,6 +634,7 @@ ff_parse_line(const char* line)
ff_parsed_line_t* result = g_malloc0(sizeof(ff_parsed_line_t));
result->file_offset = -1;
result->marked_read = -1; // unset by default
// Parse timestamp — everything up to first space followed by '['
char* bracket_start = strchr(work, '[');
@@ -648,6 +664,12 @@ ff_parse_line(const char* line)
result->archive_id = ff_unescape_meta_value(parts[i] + 4);
} else if (g_str_has_prefix(parts[i], "corrects:")) {
result->replace_id = ff_unescape_meta_value(parts[i] + 9);
} else if (g_str_has_prefix(parts[i], "to:")) {
result->to_jid = ff_unescape_meta_value(parts[i] + 3);
} else if (g_str_has_prefix(parts[i], "to_res:")) {
result->to_resource = ff_unescape_meta_value(parts[i] + 7);
} else if (g_str_has_prefix(parts[i], "read:")) {
result->marked_read = atoi(parts[i] + 5) ? 1 : 0;
}
}
g_strfreev(parts);
@@ -747,6 +769,9 @@ ff_parsed_to_profmessage(ff_parsed_line_t* pl)
ProfMessage* msg = message_init();
msg->id = pl->stanza_id ? g_strdup(pl->stanza_id) : NULL;
msg->from_jid = jid_create_from_bare_and_resource(pl->from_jid, pl->from_resource);
if (pl->to_jid) {
msg->to_jid = jid_create_from_bare_and_resource(pl->to_jid, pl->to_resource);
}
msg->plain = g_strdup(pl->message ? pl->message : "");
msg->timestamp = g_date_time_ref(pl->timestamp);
msg->type = ff_get_message_type_type(pl->type);