Results from LLM-as-a-Judge for assessing maintainability. First results with a causal approach.

These are the agreement data between the LLM and the ground truth as described in: LLM-as-a-Judge for assessing maintainability. First results with a causal approach. https://openreview.net/forum?id=Rcgr7WPr7T- filename: the name of the Java file containing the code assessed by the LLM-as-a-Judge- question_type: the type of question asked to the judge model. Should be one of: : Readability ("this code is easy to read"), Understandability ("the semantic meaning of this code is clear"), Complexity ("this code is complex"), Modularity ("this code should be broken down into smaller sections"), or Overall maintainability ("overall, this code is maintainable").- answer: the answer given by the judge model. Should be one of: "strongly agree", "weakly agree", "weakly disagree", or "strongly disagree".- explanation: the textula explanation provided by the judge model.- model: the model used (NOTE: here "llm" corresponds to the model Qwen3.5-397B-A17B running in vLLM).- temperature: the temperature value used for the- input_tokens: the number of input tokens as reported by pydantic_ai- output_tokens: the number of output tokens as reported by pydantic_ai- ground_truth: the most likely answer from human experts (from Schnappinger et al. 2020)- match_type: what type of agreement it is. Should be one of: "exact_match" (both humans and judge models have given the same answer), "partial_agree" / "partial_disagree" (Humans experts and judge models both agree (resp. disagree) but diverge in their strength (weakly vs strongly): example: human expert says "weakly agree", judge model says "strongly agree" --> partial_agree), "wrong_direction" (Humans experts and judge models have opposite opinions: one agrees the other disagrees (independant of the strenght: weakly or strongly). example human expert says "weakly agree" and the judge model says "strongly disagree").