cproof/docs/json-glib-integration-plan.md

JSON-GLib Integration Plan for Issue #15

Overview

This document outlines the complete plan for integrating json-glib into the CProof (profanity) build system and codebase. The integration replaces the strstr-based JSON parsing in _parse_ai_response() with proper json-glib parsing, and adds comprehensive unit tests.

---

Current State Analysis

What Has Already Been Done

Component	Status	Details
configure.ac	DONE	PKG_CHECK_MODULES([json-glib], [json-glib-1.0 >= 1.6.0]) added at line 101-102
src/ai/ai_client.c	DONE	#include <json-glib/json-glib.h> added at line 24; _parse_ai_response() rewritten using json-glib
Dockerfile.ubuntu	DONE	libjson-glib-dev added at line 37
Dockerfile.debian	DONE	libjson-glib-dev added at line 38

What Is Missing (Root Causes of bootstrap.sh Failure)

1. Makefile.am missing json-glib flags:

   • AM_CFLAGS at line 272 does not include $(json-glib_CFLAGS)
   • tests_unittests_unittests_LDADD at line 299 does not include $(json-glib_LIBS)
   • The main profanity binary target also needs $(json-glib_LIBS) in its link flags

2. No unit tests for _parse_ai_response():

   • test_ai_client.c has 464 lines of tests but none for the JSON parsing function
   • The function is static, so it cannot be tested directly; a test harness or refactoring is needed

3. bootstrap.sh failure:

   • bootstrap.sh is a simple script: mkdir -p m4 && autoreconf -i "$@"
   • The failure is caused by autoreconf failing due to missing json-glib m4 macros in the m4/ directory, or the configure.ac referencing PKG_CHECK_MODULES without the proper ax_pthread.m4 or json-glib pkg-config files being available in the build environment

---

Detailed Plan

1. Build System Changes

1.1 configure.ac — No Changes Needed

The json-glib check is already present:

PKG_CHECK_MODULES([json-glib], [json-glib-1.0 >= 1.6.0], [],
    [AC_MSG_ERROR([json-glib 1.6.0 or higher is required])])

This generates the variables json-glib_CFLAGS and json-glib_LIBS for use in Makefile.am.

1.2 Makefile.am — Changes Required

File: Makefile.am

Change 1: Add json-glib_CFLAGS to AM_CFLAGS (line 272)

# Current (line 272):
AM_CFLAGS = @AM_CFLAGS@ -I$(srcdir)/src

# Change to:
AM_CFLAGS = @AM_CFLAGS@ $(json-glib_CFLAGS) -I$(srcdir)/src

Change 2: Add json-glib_LIBS to profanity linking (line 275)

The profanity binary target needs json-glib_LIBS. Since profanity_SOURCES is defined at line 275 and no explicit profanity_LDADD exists, we need to add one:

# Add after line 275:
profanity_LDADD = $(json-glib_LIBS) $(LIBS)

Change 3: Add json-glib_LIBS to test linking (line 299)

# Current (line 299):
tests_unittests_unittests_LDADD = -lcmocka

# Change to:
tests_unittests_unittests_LDADD = -lcmocka $(json-glib_LIBS)

1.3 Why bootstrap.sh Fails

The bootstrap.sh script runs autoreconf -i, which regenerates the configure script from configure.ac. The failure occurs because:

1. pkg-config dependency: PKG_CHECK_MODULES requires pkg-config to be available at configure-time (not build-time). The autoreconf step itself should succeed, but the resulting configure script will fail if pkg-config --modversion json-glib-1.0 >= 1.6.0 cannot find the json-glib installation.

2. Likely root cause: The build environment (Docker container or host) does not have json-glib development files installed, or pkg-config cannot locate the json-glib-1.0.pc file.

Fix: Ensure libjson-glib-dev (Debian/Ubuntu) or json-glib (Arch/Fedora) is installed in the build environment before running bootstrap.sh.

---

2. Source Code Changes

2.1 src/ai/ai_client.c — No Changes Needed

The _parse_ai_response() function has already been rewritten using json-glib:

static gchar*
_parse_ai_response(const gchar* response_json)
{
    // ... json-glib parsing with three fallback strategies:
    //   Try 1: Perplexity /v1/agent format
    //   Try 2: OpenAI /v1/chat/completions format
    //   Try 3: Fallback "text" field
}

2.2 src/ai/ai_client.h — No Changes Needed

The header file already includes <glib.h> and defines all necessary types. No json-glib-specific types are exposed (json-glib is an implementation detail).

---

3. Test Changes

3.1 tests/unittests/test_ai_client.c — New Tests Required

The current test file has 464 lines covering provider management, session handling, JSON escaping, and autocomplete. No tests exist for _parse_ai_response() because it is static.

Option A: Refactor _parse_ai_response to be testable (Recommended)

Make the function non-static and add a test wrapper:

// In ai_client.h, add:
gchar* ai_parse_ai_response(const gchar* response_json);

// In ai_client.c, change:
// static gchar* _parse_ai_response(...) → gchar* ai_parse_ai_response(...)

Option B: Use a test harness with extern declaration

Add to test file:

extern gchar* _parse_ai_response(const gchar* response_json);

Recommended tests to add:

Test Name	Description
test_ai_parse_response_null	NULL input returns NULL
test_ai_parse_response_empty	Empty string returns NULL
test_ai_parse_response_invalid_json	Malformed JSON returns NULL
test_ai_parse_response_openai_format	Valid OpenAI /v1/chat/completions response
test_ai_parse_response_openai_empty_choices	Response with empty choices array
test_ai_parse_response_perplexity_format	Valid Perplexity /v1/agent response
test_ai_parse_response_perplexity_empty_output	Response with empty output array
test_ai_parse_response_text_fallback	Response with only "text" field
test_ai_parse_response_unicode_content	Content with unicode characters
test_ai_parse_response_multiline_content	Content with newlines and tabs
test_ai_parse_response_html_content	Content with HTML tags
test_ai_parse_response_error_format	Response with "error" field (not "choices")

3.2 tests/unittests/test_ai_client.h — Update Required

Add declarations for new test functions:

/* JSON parsing tests */
void test_ai_parse_response_null(void** state);
void test_ai_parse_response_empty(void** state);
void test_ai_parse_response_invalid_json(void** state);
void test_ai_parse_response_openai_format(void** state);
void test_ai_parse_response_openai_empty_choices(void** state);
void test_ai_parse_response_perplexity_format(void** state);
void test_ai_parse_response_perplexity_empty_output(void** state);
void test_ai_parse_response_text_fallback(void** state);
void test_ai_parse_response_unicode_content(void** state);
void test_ai_parse_response_multiline_content(void** state);
void test_ai_parse_response_html_content(void** state);
void test_ai_parse_response_error_format(void** state);

3.3 tests/unittests/unittests.c — Update Required

Add new test entries after the existing AI client tests (around line 696):

// After line 696 (test_ai_json_escape_backslash_quote):
cmocka_unit_test(test_ai_parse_response_null),
cmocka_unit_test(test_ai_parse_response_empty),
cmocka_unit_test(test_ai_parse_response_invalid_json),
cmocka_unit_test(test_ai_parse_response_openai_format),
cmocka_unit_test(test_ai_parse_response_openai_empty_choices),
cmocka_unit_test(test_ai_parse_response_perplexity_format),
cmocka_unit_test(test_ai_parse_response_perplexity_empty_output),
cmocka_unit_test(test_ai_parse_response_text_fallback),
cmocka_unit_test(test_ai_parse_response_unicode_content),
cmocka_unit_test(test_ai_parse_response_multiline_content),
cmocka_unit_test(test_ai_parse_response_html_content),
cmocka_unit_test(test_ai_parse_response_error_format),

---

4. Dockerfile Changes

4.1 Dockerfile.ubuntu — No Changes Needed

libjson-glib-dev is already present at line 37.

4.2 Dockerfile.debian — No Changes Needed

libjson-glib-dev is already present at line 38.

4.3 Dockerfile.fedora — Check Required

Verify json-glib-devel is installed. If not, add:

json-glib-devel \

4.4 Dockerfile.arch — Check Required

Verify json-glib is installed. If not, add:

json-glib \

---

5. Bootstrap.sh Fix

5.1 Current State

bootstrap.sh:

#!/bin/sh
mkdir -p m4
autoreconf -i "$@"

5.2 Root Cause Analysis

The script itself is correct. The failure is due to one of:

1. Missing json-glib-1.0.pc: pkg-config cannot find json-glib in the build environment
2. Missing m4/ax_pthread.m4: The configure.ac uses ACX_PTHREAD which requires the ax_pthread.m4 macro
3. Missing m4/ macros: Other autoconf macros referenced in configure.ac are not present

5.3 Fix Steps

1. Ensure json-glib-dev is installed in the build environment
2. Verify m4 macros exist:

   ls -la m4/
   # Should contain: ax_pthread.m4, ax_valgrind_check.m4, etc.
   

3. If m4 macros are missing, they need to be copied from the system:

   # For Debian/Ubuntu:
   cp /usr/share/autoconf-archive/m4/ax_pthread.m4 m4/
   cp /usr/share/autoconf-archive/m4/ax_valgrind_check.m4 m4/
   

4. Test bootstrap.sh manually:

   ./bootstrap.sh
   ./configure
   make
   

---

Implementation Order

graph TD
    A[1. Fix Makefile.am] --> B[2. Verify bootstrap.sh works]
    B --> C[3. Add test functions to test_ai_client.c]
    C --> D[4. Update test_ai_client.h]
    D --> E[5. Register tests in unittests.c]
    E --> F[6. Verify Dockerfiles have json-glib-dev]
    F --> G[7. Run make check to verify all tests pass]

---

Summary of Changes

File	Change Type	Lines Affected
Makefile.am	Modify	Line 272 (AM_CFLAGS), Line 275 (add profanity_LDADD), Line 299 (LDADD)
src/ai/ai_client.c	None	Already complete
src/ai/ai_client.h	None	Already complete
tests/unittests/test_ai_client.c	Add	~200 lines of new test functions
tests/unittests/test_ai_client.h	Add	~12 new declarations
tests/unittests/unittests.c	Add	~12 new test entries
Dockerfile.ubuntu	None	Already has libjson-glib-dev
Dockerfile.debian	None	Already has libjson-glib-dev
Dockerfile.fedora	Check	Verify json-glib-devel present
Dockerfile.arch	Check	Verify json-glib present
bootstrap.sh	Diagnose	Fix is environmental, not script change

---

Verification Checklist

• Makefile.am updated with $(json-glib_CFLAGS) and $(json-glib_LIBS)
• bootstrap.sh runs successfully
• ./configure completes without json-glib errors
• make compiles without errors
• make check runs all tests including new json-glib tests
• All existing tests still pass
• Docker builds succeed with json-glib integration