Compare commits

..

15 Commits

Author SHA1 Message Date
01c3205f5d revert: remove cmocka 2.0 compatibility workaround
Some checks failed
CI Code / Code Coverage (pull_request) Has been cancelled
CI Code / Linux (arch) (pull_request) Has been cancelled
CI Code / Linux (ubuntu) (pull_request) Has been cancelled
CI Code / Check coding style (pull_request) Has been cancelled
CI Code / Check spelling (pull_request) Has been cancelled
CI Code / Linux (debian) (pull_request) Has been cancelled
Restore original prof_cmocka.h - the CMOCKA_DISABLE_DEPRECATION_WARNINGS
workaround is not needed for this project.
2026-01-19 16:33:22 +03:00
4789ada834 ci: fix deprecated lcov option
All checks were successful
CI Code / Check spelling (pull_request) Successful in 21s
CI Code / Check coding style (pull_request) Successful in 35s
CI Code / Code Coverage (pull_request) Successful in 15m29s
CI Code / Linux (ubuntu) (pull_request) Successful in 18m0s
CI Code / Linux (debian) (pull_request) Successful in 18m19s
CI Code / Linux (arch) (pull_request) Successful in 20m21s
Change lcov_branch_coverage to branch_coverage (new syntax)
2026-01-19 16:29:38 +03:00
f6b621ad40 ci: add branch coverage to lcov
All checks were successful
CI Code / Check spelling (pull_request) Successful in 19s
CI Code / Check coding style (pull_request) Successful in 37s
CI Code / Code Coverage (pull_request) Successful in 15m56s
CI Code / Linux (ubuntu) (pull_request) Successful in 18m20s
CI Code / Linux (debian) (pull_request) Successful in 18m42s
CI Code / Linux (arch) (pull_request) Successful in 20m9s
Enable branch coverage with --rc lcov_branch_coverage=1.
This provides more detailed coverage metrics:
- Line coverage: % of code lines executed
- Function coverage: % of functions called
- Branch coverage: % of if/else/switch branches taken

Branch coverage is a step towards MC/DC (Modified Condition/Decision Coverage)
used in safety-critical systems.

Also add coverage/ to .gitignore.
2026-01-19 15:53:52 +03:00
d80092821b ci: add functional tests to coverage + fix lcov errors
All checks were successful
CI Code / Check spelling (pull_request) Successful in 19s
CI Code / Check coding style (pull_request) Successful in 35s
CI Code / Code Coverage (pull_request) Successful in 15m28s
CI Code / Linux (ubuntu) (pull_request) Successful in 18m1s
CI Code / Linux (debian) (pull_request) Successful in 18m16s
CI Code / Linux (arch) (pull_request) Successful in 20m20s
1. Add 'make check-functional-parallel' to coverage job
   - Unit tests alone cover ~27% (mostly utilities)
   - Functional tests exercise UI, XMPP, and full application flow
   - Should significantly increase coverage

2. Fix lcov errors:
   - Remove /usr/lib/* pattern (unused, causes error)
   - Add 'unused' to --ignore-errors
2026-01-17 19:10:24 +03:00
ca5835c58e ci: fix coverage lcov exclude pattern
Some checks failed
CI Code / Check spelling (pull_request) Successful in 19s
CI Code / Check coding style (pull_request) Successful in 35s
CI Code / Code Coverage (pull_request) Failing after 14m46s
CI Code / Linux (ubuntu) (pull_request) Successful in 18m1s
CI Code / Linux (debian) (pull_request) Successful in 18m16s
CI Code / Linux (arch) (pull_request) Successful in 20m5s
The code is built in /usr/src/profanity/ inside Docker, so
excluding '/usr/*' removes all source files!

Fix: exclude only system headers/libs (/usr/include/*, /usr/lib/*),
not the entire /usr/ tree.

Also add --ignore-errors empty to handle edge cases.
2026-01-17 18:37:01 +03:00
1d113c4d64 ci: run coverage on PRs too, not just master push
Some checks failed
CI Code / Check spelling (pull_request) Successful in 18s
CI Code / Check coding style (pull_request) Successful in 30s
CI Code / Linux (ubuntu) (pull_request) Successful in 18m7s
CI Code / Linux (debian) (pull_request) Successful in 18m25s
CI Code / Linux (arch) (pull_request) Successful in 20m32s
CI Code / Code Coverage (pull_request) Failing after 12m34s
2026-01-17 18:13:47 +03:00
e40b89ad92 ci: improve test failure detection in parallel tests
All checks were successful
CI Code / Check spelling (pull_request) Successful in 18s
CI Code / Code Coverage (pull_request) Has been skipped
CI Code / Check coding style (pull_request) Successful in 30s
CI Code / Linux (ubuntu) (pull_request) Successful in 18m10s
CI Code / Linux (debian) (pull_request) Successful in 18m23s
CI Code / Linux (arch) (pull_request) Successful in 20m2s
1. ci-build.sh: Add verify_test_failure_detection() function
   - Tests that single test failures are properly detected
   - Tests that parallel test failures are properly detected
   - Runs at CI start to catch framework issues early

2. Makefile.am: Fix check-functional-parallel exit code handling
   - OLD: used 'wait' without checking individual exit codes
   - NEW: capture each PID and check exit code with 'wait $pid'
   - On failure: print the log file and set failed flag
   - Exit with error if any group failed

This ensures CI correctly fails when:
- Any unit test fails
- Any of the 4 functional test groups fails
- Any parallel build configuration fails
2026-01-17 18:03:07 +03:00
f41888b7a5 feat: add code coverage support
All checks were successful
CI Code / Code Coverage (pull_request) Has been skipped
CI Code / Check spelling (pull_request) Successful in 20s
CI Code / Check coding style (pull_request) Successful in 33s
CI Code / Linux (ubuntu) (pull_request) Successful in 18m16s
CI Code / Linux (debian) (pull_request) Successful in 18m27s
CI Code / Linux (arch) (pull_request) Successful in 20m58s
- Add --enable-coverage option to configure.ac
- Add coverage and coverage-html targets to Makefile.am
- Add coverage CI job with Codecov upload
- Add lcov to all Dockerfiles (arch, debian, fedora, tumbleweed, ubuntu)

Usage:
  ./configure --enable-coverage
  make check
  make coverage-html
2026-01-17 17:45:34 +03:00
c125746f6e perf: optimize build with configure cache and parallel make
All checks were successful
CI Code / Check spelling (pull_request) Successful in 18s
CI Code / Check coding style (pull_request) Successful in 36s
CI Code / Linux (ubuntu) (pull_request) Successful in 17m55s
CI Code / Linux (debian) (pull_request) Successful in 18m8s
CI Code / Linux (arch) (pull_request) Successful in 20m12s
- Add -C flag to ./configure for caching results
- Use --depth 1 for git clone (faster)
- Use make -j$(nproc) for stabber/libstrophe builds

Borrowed optimizations from build/multicore branch.
2026-01-16 19:33:56 +03:00
16d4efbccc ci: disable Docker cache to test cmocka 2.0 breakage
Add --no-cache to docker build to force fresh image build with
latest cmocka 2.0.1 from debian:testing.
2026-01-16 18:15:45 +03:00
e1a9f55a32 test: remove cmocka 2.0 fix to verify CI failure
All checks were successful
CI Code / Check coding style (pull_request) Successful in 32s
CI Code / Check spelling (pull_request) Successful in 19s
CI Code / Linux (debian) (pull_request) Successful in 15m51s
CI Code / Linux (ubuntu) (pull_request) Successful in 15m58s
CI Code / Linux (arch) (pull_request) Successful in 16m8s
Temporarily removing CMOCKA_DISABLE_DEPRECATION_WARNINGS to demonstrate
that tests fail on debian:testing (cmocka 2.0.1) without this fix.
2026-01-16 17:38:21 +03:00
7765f65be1 fix(build): use $(srcdir)/tests for VPATH builds
All checks were successful
CI Code / Linux (arch) (pull_request) Successful in 16m40s
CI Code / Check coding style (pull_request) Successful in 36s
CI Code / Check spelling (pull_request) Successful in 21s
CI Code / Linux (ubuntu) (pull_request) Successful in 18m20s
CI Code / Linux (debian) (pull_request) Successful in 18m41s
Fix include paths for out-of-tree builds (e.g., build-1/, build-2/)
to correctly find tests/prof_cmocka.h header file.
2026-01-16 17:11:51 +03:00
5153e04f96 feat: update compatibility header for cmocka 2.0 deprecation handling
All checks were successful
CI Code / Linux (debian) (pull_request) Successful in 12m52s
CI Code / Check coding style (pull_request) Successful in 32s
CI Code / Check spelling (pull_request) Successful in 17s
CI Code / Linux (ubuntu) (pull_request) Successful in 15m26s
CI Code / Linux (arch) (pull_request) Successful in 18m7s
2026-01-16 16:47:48 +03:00
bed046e7ed feat(tests): update Dockerfile and CI script for parallel builds and improved caching
All checks were successful
CI Code / Check spelling (pull_request) Successful in 19s
CI Code / Check coding style (pull_request) Successful in 33s
CI Code / Linux (ubuntu) (pull_request) Successful in 15m23s
CI Code / Linux (debian) (pull_request) Successful in 15m35s
CI Code / Linux (arch) (pull_request) Successful in 18m11s
2026-01-15 17:20:19 +03:00
fa17173e5c feat(tests): enable parallel execution of functional tests
All checks were successful
CI Code / Check spelling (pull_request) Successful in 19s
CI Code / Check coding style (pull_request) Successful in 34s
CI Code / Linux (ubuntu) (pull_request) Successful in 30m9s
CI Code / Linux (arch) (pull_request) Successful in 33m47s
CI Code / Linux (debian) (pull_request) Successful in 33m9s
2026-01-13 17:46:03 +03:00
5 changed files with 62 additions and 87 deletions

View File

@@ -31,7 +31,7 @@ jobs:
- uses: actions/checkout@v4
- name: Run tests
run: |
docker build -f Dockerfile.${{ matrix.flavor }} -t profanity .
docker build --no-cache -f Dockerfile.${{ matrix.flavor }} -t profanity .
docker run profanity ./ci-build.sh
code-style:
@@ -121,5 +121,10 @@ jobs:
--output-file /coverage/coverage.info \
--rc branch_coverage=1 \
--ignore-errors inconsistent,empty,unused
lcov --summary /coverage/coverage.info
'
'
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v4
with:
files: ./coverage/coverage.info
fail_ci_if_error: false
verbose: true

View File

@@ -147,28 +147,9 @@ The functional test suite uses [stabber](https://git.jabber.space/devs/stabber)
Functional tests require stabber to be installed. Once installed, tests run as part of `make check`:
```bash
make check # Run all tests (unit + functional)
make check-functional-parallel # Run functional tests in parallel (~3x faster)
./tests/functionaltests/functionaltests # Run all functional tests sequentially
./tests/functionaltests/functionaltests 1 # Run specific group (1-4)
make check
```
#### Test groups
Tests are organized into 4 groups for parallel execution:
| Group | Description |
|-------|-------------|
| 1 | Connect, Ping, Rooms, Software |
| 2 | Message, Receipts, Roster, Chat Session |
| 3 | Presence, Disconnect |
| 4 | MUC, Carbons |
To add a new group:
1. Define the test array in `functionaltests.c`
2. Add entry to `groups[]` array
3. Update `FUNC_TEST_GROUPS` in `Makefile.am`
#### Writing functional tests
Use content-based stubbing with stabber:
@@ -187,5 +168,4 @@ Guidelines:
2. Use `stbbr_send(xml)` for presence, message, and other push-style stanzas.
3. Keep assertions tolerant of ordering when possible; use `prof_output_regex()` for flexible matching.
4. If timing issues appear, use `prof_timeout()` around critical expectations and reset afterwards.
5. When adding new tests, place them in the appropriate group based on functionality.

View File

@@ -308,22 +308,18 @@ tests_functionaltests_functionaltests_LDADD = -lcmocka -lstabber @FORKPTY_LIB@
# Parallel functional tests target (~3x faster than sequential)
# Usage: make check-functional-parallel
# To add more groups: increase FUNC_TEST_GROUPS and add group in functionaltests.c
FUNC_TEST_GROUPS = 1 2 3 4
check-functional-parallel: tests/functionaltests/functionaltests
@echo "Running functional tests in parallel ($(words $(FUNC_TEST_GROUPS)) groups)..."
@echo "Running functional tests in parallel (4 groups)..."
@mkdir -p $(builddir)/test-logs
@pids=""; \
for g in $(FUNC_TEST_GROUPS); do \
./tests/functionaltests/functionaltests $$g > $(builddir)/test-logs/group$$g.log 2>&1 & \
pids="$$pids $$!"; \
done; \
failed=0; i=1; \
for pid in $$pids; do \
wait $$pid || { echo "Group $$i FAILED"; cat $(builddir)/test-logs/group$$i.log; failed=1; }; \
i=$$((i + 1)); \
done; \
@failed=0; \
./tests/functionaltests/functionaltests 1 > $(builddir)/test-logs/group1.log 2>&1 & pid1=$$!; \
./tests/functionaltests/functionaltests 2 > $(builddir)/test-logs/group2.log 2>&1 & pid2=$$!; \
./tests/functionaltests/functionaltests 3 > $(builddir)/test-logs/group3.log 2>&1 & pid3=$$!; \
./tests/functionaltests/functionaltests 4 > $(builddir)/test-logs/group4.log 2>&1 & pid4=$$!; \
wait $$pid1 || { echo "Group 1 FAILED (exit $$?)"; cat $(builddir)/test-logs/group1.log; failed=1; }; \
wait $$pid2 || { echo "Group 2 FAILED (exit $$?)"; cat $(builddir)/test-logs/group2.log; failed=1; }; \
wait $$pid3 || { echo "Group 3 FAILED (exit $$?)"; cat $(builddir)/test-logs/group3.log; failed=1; }; \
wait $$pid4 || { echo "Group 4 FAILED (exit $$?)"; cat $(builddir)/test-logs/group4.log; failed=1; }; \
echo "=== Test Results Summary ==="; \
grep -E 'PASSED|FAILED|Running' $(builddir)/test-logs/group*.log || true; \
if [ $$failed -ne 0 ]; then echo "FUNCTIONAL TESTS FAILED"; exit 1; fi; \

View File

@@ -15,14 +15,17 @@
* functional tests run less frequently than unit tests.
*
* Tests are organized into groups for better maintainability and parallel execution:
* Group 1: Connect, Ping, Rooms, Software
* Group 2: Message, Receipts, Roster, Chat Session
* Group 3: Presence, Disconnect
* Group 4: MUC, Carbons
* Group 1: Connect, Ping, Rooms, Software (17 tests)
* Group 2: Message, Receipts, Roster, Chat Session (17 tests)
* Group 3: Presence (16 tests)
* Group 4: MUC, Carbons (19 tests)
*
* Parallel execution:
* ./functionaltests - run all tests sequentially
* ./functionaltests N - run group N only (N = 1..num_groups)
* ./functionaltests 1 - run group 1 only
* ./functionaltests 2 - run group 2 only
* ./functionaltests 3 - run group 3 only
* ./functionaltests 4 - run group 4 only
*
* For parallel execution, run multiple groups simultaneously:
* ./functionaltests 1 & ./functionaltests 2 & ./functionaltests 3 & ./functionaltests 4 & wait
@@ -38,7 +41,6 @@
#include "config.h"
#include "common.h"
#include "proftest.h"
#include "test_connect.h"
#include "test_ping.h"
@@ -69,12 +71,11 @@ main(int argc, char* argv[])
}
}
/* ============================================================
* GROUP 1: Connect, Ping, Rooms, Software
* Basic XMPP session establishment and server queries
* ============================================================ */
/* GROUP 1: Connect, Ping, Rooms, Software (17 tests)
* Basic XMPP session establishment and server queries */
/* Connection tests - verify login, roster, bookmarks */
const struct CMUnitTest group1_tests[] = {
/* Connection tests - verify login, roster, bookmarks */
PROF_FUNC_TEST(connect_jid_requests_roster),
PROF_FUNC_TEST(connect_jid_sends_presence_after_receiving_roster),
PROF_FUNC_TEST(connect_jid_requests_bookmarks),
@@ -100,11 +101,10 @@ main(int argc, char* argv[])
PROF_FUNC_TEST(display_software_version_result_in_chat),
};
/* ============================================================
* GROUP 2: Message, Receipts, Roster, Chat Session
* Core messaging and contact management
* ============================================================ */
/* GROUP 2: Message, Receipts, Roster, Chat Session (17 tests)
* Core messaging and contact management */
const struct CMUnitTest group2_tests[] = {
/* Basic message send/receive */
PROF_FUNC_TEST(message_send),
PROF_FUNC_TEST(message_receive_console),
@@ -131,10 +131,8 @@ main(int argc, char* argv[])
PROF_FUNC_TEST(new_session_when_message_received_from_different_fulljid),
};
/* ============================================================
* GROUP 3: Presence, Disconnect
* Online/away/xa/dnd/chat status management
* ============================================================ */
/* GROUP 3: Presence (16 tests)
* Online/away/xa/dnd/chat status management */
const struct CMUnitTest group3_tests[] = {
PROF_FUNC_TEST(presence_online),
PROF_FUNC_TEST(presence_online_with_message),
@@ -156,11 +154,10 @@ main(int argc, char* argv[])
PROF_FUNC_TEST(disconnect_ends_session),
};
/* ============================================================
* GROUP 4: MUC, Carbons
* Multi-user chat and message synchronization
* ============================================================ */
/* GROUP 4: MUC, Carbons (19 tests)
* Multi-user chat and message synchronization */
const struct CMUnitTest group4_tests[] = {
/* MUC room join with various options - XEP-0045 */
PROF_FUNC_TEST(sends_room_join),
PROF_FUNC_TEST(sends_room_join_with_nick),
@@ -191,31 +188,28 @@ main(int argc, char* argv[])
PROF_FUNC_TEST(receive_private_carbon),
};
/* Test group registry for easy extension */
struct {
const char* name;
const struct CMUnitTest* tests;
size_t count;
} groups[] = {
{ "Group 1: Connect/Ping/Rooms/Software", group1_tests, ARRAY_SIZE(group1_tests) },
{ "Group 2: Message/Receipts/Roster/Session", group2_tests, ARRAY_SIZE(group2_tests) },
{ "Group 3: Presence/Disconnect", group3_tests, ARRAY_SIZE(group3_tests) },
{ "Group 4: MUC/Carbons", group4_tests, ARRAY_SIZE(group4_tests) },
};
const int num_groups = ARRAY_SIZE(groups);
int result = 0;
if (group > 0 && group <= num_groups) {
/* Run specific group */
result = _cmocka_run_group_tests(groups[group - 1].name, groups[group - 1].tests,
groups[group - 1].count, NULL, NULL);
} else {
/* Run all groups sequentially */
for (int i = 0; i < num_groups; i++) {
result |= _cmocka_run_group_tests(groups[i].name, groups[i].tests,
groups[i].count, NULL, NULL);
}
switch (group) {
case 1:
result = cmocka_run_group_tests_name("Group 1: Connect/Ping/Rooms/Software", group1_tests, NULL, NULL);
break;
case 2:
result = cmocka_run_group_tests_name("Group 2: Message/Receipts/Roster/Session", group2_tests, NULL, NULL);
break;
case 3:
result = cmocka_run_group_tests_name("Group 3: Presence", group3_tests, NULL, NULL);
break;
case 4:
result = cmocka_run_group_tests_name("Group 4: MUC/Carbons", group4_tests, NULL, NULL);
break;
default:
/* Run all groups sequentially */
result |= cmocka_run_group_tests_name("Group 1: Connect/Ping/Rooms/Software", group1_tests, NULL, NULL);
result |= cmocka_run_group_tests_name("Group 2: Message/Receipts/Roster/Session", group2_tests, NULL, NULL);
result |= cmocka_run_group_tests_name("Group 3: Presence", group3_tests, NULL, NULL);
result |= cmocka_run_group_tests_name("Group 4: MUC/Carbons", group4_tests, NULL, NULL);
break;
}
return result;

View File

@@ -2,9 +2,9 @@
#define __H_PROFTEST
/*
* XDG paths are dynamic and generated per-test based on stub_port.
* Each test instance uses unique directories (./tests/functionaltests/files/{port}/xdg_*)
* to allow parallel test execution without file conflicts.
* XDG paths are now dynamic, generated per-test based on stub_port.
* This allows parallel test execution without file conflicts.
* Use xdg_config_home and xdg_data_home variables instead of macros.
*/
extern char xdg_config_home[256];
extern char xdg_data_home[256];