profanity/baseline.csv at b1d820462a61c7cc41cdf987aec630c854453b5e

mirror of https://git.jabber.space/devs/cproof.git synced 2026-07-19 18:46:21 +00:00

Files

jabber.developer2 b1d820462a

feat(bench): synthetic load harness for flat-file backend (P1-P5)

End-to-end performance and correctness harness for the flat-file +
SQLite database backends. Lives in tests/bench/, built only on
demand (`make bench`); not part of `make check`.

Components

  gen_history (P1)
    Deterministic corpus generator. Knobs: lines, contacts, years,
    seed, stanza-id mode (uuid/libpurple/conversations/mixed), LMC
    rate, MAM-OOO rate, resources/contact, length profile
    (short/mixed/long/extreme). Emits the canonical
    flatlog/<account>/<contact>/history.log layout used by
    ff_verify_integrity. ~340 LOC.

  bench_runner (P1, P2.5)
    S1 cold tail-access via sparse index
    S2 warm tail-access (page cache hot)
    S3 deep pagination (1000 binary-search lookups)
    S4 first-time index build (cold file -> ff_state_ensure_fresh)
    S5 incremental extend (asserts no full rebuild path)
    S6 real ff_verify_integrity over the contact tree
    Reports total/err/warn/info issue counts in the CSV note.

  bench_long_messages (P2)
    L1-L14 long-message stress: 1KB up to 9.9MB bodies, plus
    oversized line rejection (10MB+1 -> ff_readline returns ""),
    embedded-newline / pipe / emoji body patterns, full parse on
    100x1MB, 1000x100KB sustained append.

  bench_failure_modes (P3)
    F1-F15 failure-injection: truncated last line, mid-file CRLF,
    mid-file BOM, LMC cycle, LMC depth>FF_MAX_LMC_DEPTH, manual
    ': ' in resource, RTL/ZWSP, Latin-1 byte, empty body,
    mtime/inode flip, empty file. Each test asserts expected issue
    levels and reports PASS/FAIL.

  bench_export_import (P5)
    Links real database_export.c + database_sqlite.c + database.c
    and drives log_database_export_to_flatfile /
    log_database_import_from_flatfile under load.
    Subcommands: seed, export, import, roundtrip, verify.
    S7a/b export, S8a/b import, S8e roundtrip with full byte-by-byte
    content diff of every row in (from_jid, to_jid, message,
    timestamp, type, stanza_id, archive_id, encryption, replace_id).

Make targets

  bench-quick / bench / bench-full
  bench-longmsg, bench-failure
  bench-multicontact, bench-lmc, bench-ooo
  bench-export, bench-import, bench-roundtrip
  bench-pipeline, bench-pipeline-max (1M rows)
  bench-compare, bench-update-baseline

Volume controls: BENCH_VOLUME (small/medium/max), BENCH_PIPE_ROWS,
BENCH_PIPE_ROWS_MAX, BENCH_DATA_DIR, BENCH_CSV.

Baseline + regression checking (P4)

  tests/bench/baseline.csv       51 rows: S1-S6 x {small,lmc,ooo}
                                 + L1-L14 + F1-F15 (11 of 15) +
                                 S7/S8 x {pipe100k, pipe1M}.
  compare_baseline.py            median over duplicate rows;
                                 exits 1 on any (scenario, volume)
                                 slowdown >= threshold (default 25%).

Verified at scale

  bench-pipeline-max: 1,000,000 rows, full content diff
    seed   17 s, export 304 s, import 31 s, idempotent re-import 10 s,
    diff 2.7 s -- mismatches=0.

Findings surfaced by the harness

  Export scales super-linearly: 4 s @ 100k -> 304 s @ 1M (76x for
    10x rows). Cause: g_slist_sort on the merged list + per-row
    ProfMessage/ff_parsed_line_t allocations. RSS peaks at 1.4 GB
    on 1M. Worth a follow-up.
  Export progress reporting only fires during the write phase --
    the merge+sort phase (~95% of wall time at 1M) is silent.
  /history export and /history import are blocking on the main
    UI thread; profanity is frozen for the duration (~5 min @ 1M).
  ff_readline sets *truncated=TRUE on partial-write tail but
    ff_verify_integrity does not surface this -- partial writes
    go unflagged (failure-injection F1).
  Parser silently truncates body at first unescaped ': ' if a
    resource was manually edited to contain it (F9).
  In gen_history (caught by the bench's own real-verify pass):
    g_strndup mid-codepoint truncation on UTF-8 bank strings -- fixed.

Linkage strategy

  database_flatfile.c + parser + verify + common.c are linked
  unconditionally. The export/import bench additionally links
  database.c + database_sqlite.c + database_export.c. bench_stubs.c
  provides minimal stubs for log_*, prefs_*, connection_get_jid,
  jid_create, files_*, message_*, ui hooks. integrity_issue_free
  is a weak symbol so it falls back to the real database.c
  implementation when that file is linked.

2026-04-30 15:39:30 +03:00

5.0 KiB

Raw Blame History

1	scenario	volume	bytes	lines	wall_ms	peak_rss_kb	note
2	S1_cold_tail	small	39494716	10002	46.040	14408	tail page=100
3	S2_warm_tail	small	39494716	10002	33.878	14716	tail page=100
4	S3_deep_pagination	small	39494716	10002	1.024	14716	n_pages=1000
5	S4_first_build	small	39494716	10002	39.139	14716	idx_entries=20
6	S5_incremental_extend	small	104411	1000	0.859	14716	appended=1000_lines
7	S6_verify	small	39494716	10002	396.022	16424	total=1 err=0 warn=1 info=0
8	L1	longmsg	110586	100	1.486	9432	n=100 body=1024 write=0.8ms read=0.6ms parsed=100 mismatch=0 parse_fail=0 1KB body x100
9	L2	longmsg	1032186	100	10.468	9432	n=100 body=10240 write=5.7ms read=4.8ms parsed=100 mismatch=0 parse_fail=0 10KB body x100
10	L3	longmsg	10248186	100	100.337	9816	n=100 body=102400 write=57.3ms read=43.1ms parsed=100 mismatch=0 parse_fail=0 100KB body x100
11	L4	longmsg	52432936	50	502.785	13608	n=50 body=1048576 write=264.5ms read=238.3ms parsed=50 mismatch=0 parse_fail=0 1MB body x50
12	L5	longmsg	52429696	10	518.558	27764	n=10 body=5242880 write=264.4ms read=254.2ms parsed=10 mismatch=0 parse_fail=0 5MB body x10
13	L6	longmsg	41435552	4	407.803	45852	n=4 body=10358784 write=211.3ms read=196.5ms parsed=4 mismatch=0 parse_fail=0 9.9MB body x4 (just under FF_MAX_LINE_LEN)
14	L7	longmsg	10485981	2	3.750	45852	oversize=10485761_rejected=yes read=3.8ms parsed=1
15	L8	longmsg	5175286	50	45.549	45852	n=50 body=102400 write=26.6ms read=18.9ms parsed=50 mismatch=0 parse_fail=0 100KB body w/ \n every 100B
16	L9	longmsg	5124136	50	45.487	45852	n=50 body=102400 write=24.5ms read=21.0ms parsed=50 mismatch=0 parse_fail=0 100KB body w/ pipes
17	L10	longmsg	5124136	50	43.254	45852	n=50 body=102400 write=18.9ms read=24.4ms parsed=50 mismatch=0 parse_fail=0 100KB body utf-8 emoji
18	L11	longmsg	110586	100	1.284	45852	n=100 body=1024 write=0.7ms read=0.6ms parsed=100 mismatch=0 parse_fail=0 sanity baseline
19	L12	longmsg	5415366	1000	26.437	45852	5x1MB_in_last_100 parsed=1000
20	L13	longmsg	104865786	100	932.630	45852	n=100 body=1048576 write=485.7ms read=446.9ms parsed=100 mismatch=0 parse_fail=0 verify-equiv full parse on 100x1MB
21	L14	longmsg	102481986	1000	962.342	45852	n=1000 body=102400 write=510.4ms read=452.0ms parsed=1000 mismatch=0 parse_fail=0 rapid append 1000x100KB
22	F1	fail-pass	904	0	0.118	9540	issues total=1 err=0 warn=1 (partial-line tail; expect graceful handle)
23	F2	fail-pass	1119	0	0.113	9540	warnings=2 (expect CRLF warning) first_warn=File permissions are 644 expected 600 (sensitive data)
24	F3	fail-pass	341	0	0.072	9540	errors=1 warnings=1 (expect 1 unparsable line for mid-file BOM) first=Unparsable line
25	F7	fail-pass	527	0	0.082	9540	issues total=1 err=0 (cycle handled at read-time not verify)
26	F8	fail-pass	19044	0	0.981	9540	201 lines (1 orig + 200 corrections) errors=0 warns=1
27	F9	fail-pass	341	0	0.069	9540	errors=0 (parser splits at first ': ' — not flagged but body truncated)
28	F10	fail-pass	176	0	0.060	9540	errors=0 warnings=1 (RTL/ZWSP preserved literally)
29	F11	fail-pass	323	0	0.264	9540	errors=1 (Latin-1 byte: error or fallback OK)
30	F12	fail-pass	330	0	0.071	9540	errors=0 (empty body OK)
31	F14	fail-pass	23236	0	0.914	9540	v1_lines=100 v2_lines=250 (expect rebuild detected and 250 lines)
32	F15	fail-pass	0	0	0.121	9540	errors=0 infos=1 warnings=1 (empty file should yield INFO)
33	S1_cold_tail	lmc	389183687	100002	480.428	43456	tail page=100
34	S2_warm_tail	lmc	389183687	100002	477.482	43456	tail page=100
35	S3_deep_pagination	lmc	389183687	100002	1.066	43456	n_pages=1000
36	S4_first_build	lmc	389183687	100002	473.604	43456	idx_entries=200
37	S5_incremental_extend	lmc	104411	1000	0.902	43456	appended=1000_lines
38	S6_verify	lmc	389183687	100002	3892.137	43456	total=1 err=0 warn=1 info=0
39	S6_verify	ooo	397586720	100002	4037.058	33336	total=17573 err=0 warn=17573 info=0
40	S7a_export_cold	pipe100000	40714240	100000	3925.249	148432	exported=100000 db_rows=100000
41	S7b_export_dedup	pipe100000	40714240	100000	3884.913	252396	exported=0 db_rows=100000
42	S7_seed_export	pipe100000	40714240	100000	3894.644	148320	exported=100000 db_rows=100000
43	S8a_import_cold	pipe100000	40628224	100000	3162.988	123212	imported=100000 rows_before=0 rows_after=100000
44	S8b_import_idempotent	pipe100000	40628224	0	1178.929	194896	imported=0 rows_before=100000 rows_after=100000
45	S8e_roundtrip	pipe100000	40521728	100000	8905.553	148340	rows=100000 seed=1640ms export=3730ms import=3258ms diff=277ms exported=100000 imported=100000 rows_a=100000 rows_b=100000 mismatches=0 body=0 lmc=0
46	S7a_export_cold	pipe1000000	407732224	1000000	309245.302	1376872	exported=1000000 db_rows=1000000
47	S7b_export_dedup	pipe1000000	407732224	1000000	303526.638	2418088	exported=0 db_rows=1000000
48	S7_seed_export	pipe1000000	407732224	1000000	304160.043	1377084	exported=1000000 db_rows=1000000
49	S8a_import_cold	pipe1000000	406605824	1000000	31288.288	1124984	imported=1000000 rows_before=0 rows_after=1000000
50	S8b_import_idempotent	pipe1000000	406605824	0	10418.850	1841644	imported=0 rows_before=1000000 rows_after=1000000
51	S8e_roundtrip	pipe1000000	405757952	1000000	352840.026	1377148	rows=1000000 seed=17637ms export=301062ms import=31430ms diff=2711ms exported=1000000 imported=1000000 rows_a=1000000 rows_b=1000000 mismatches=0 body=0 lmc=0

5.0 KiB Raw Blame History

5.0 KiB

Raw Blame History