Recommending Relevant Classes for Infrequent API Classes

Project summary As APIs are many and complicated, it is difficult to learn their usages, and programmers can introduce API-related bugs in their code. To assist programming and reduce API-related bugs, in the literature, researchers have proposed various approaches that assist programming with APIs, and many approaches need API patterns that are mined from clients or documents. However, it is rather challenging to mine complete API patterns. The incompleteness of mined patterns is a barrier for many API-related research. For example, if a tool determines that the violations of known API patterns are bugs, the tool can produce many false alarms, when API patterns are incomplete. Our study shows that from clients, it is infeasible to mine frequent patterns for many APIs, especially when their versions are considered. From documents, many API patterns are not explicitly described either. As a result, it is quite challenging to improve the completeness of API patterns. In this project, we propose a novel research direction to improve the completeness of API patterns, and it can mine patterns that do not appear in clients or documents. From clients, it is typically able to mine only limited API patterns. Our idea is to generate training data from mined patterns and documents, and to learn a model that can predict more infrequent patterns. To show the feasibility of this research direction, we take a classical type of API patterns, relevant APIs, as an example. For an API class, an relevant API is a set of API classes that are called with this API class. The prior approaches mine relevant APIs with association mining. From mined frequent call sets, our tool, APIRel, generates positive instances and negative instances, and extract their features by analyzing API documents. In this way, it trains models that can predict infrequent relevant APIs that do not appear in known clients. Setting In our evaluation, we select the APIs of accumulo, cassandra, karaf, lucene, and poi. Their API documents are listed as follows: accumulo: https://tohidemyname.github.io/accumulodoc/ cassandra: https://tohidemyname.github.io/cassandradoc/ karaf: https://tohidemyname.github.io/karafdoc/ lucene: https://tohidemyname.github.io/lucenedoc/ poi: https://tohidemyname.github.io/poidoc/ Artifacts Overview manual_sample contains about 700 manually labeled API class pairs (about 140 pairs per library across five Apache projects). Each pair was inspected by reading the official API documentation of both classes and checking whether they can be called together to implement a specific functionality. For each class pair, we focused on: Constructor and method signatures that explicitly reference the other class. Usage descriptions that imply a coordination between the two classes. Semantically aligned responsibilities in the class-level documentation. manual_sample is organized as txt, each line's first two items are class names, and the last two items are the tool prediction and the manual label. LLM_tool provides the scripts used to interact with the evaluated LLM baselines: DeepSeek-V3,Qwen3-Coder, and GPT-4o. All model responses are recorded in a structured JSON format. Result LLM-based Evaluation We reports the raw outputs of the large language model (LLM) baselines used in RQ1. The directory LLM_result contains the model predictions for all sample class pairs, organized by model and library. Each file is a JSON list of prediction records. A typical entry has the following format: { "class1": "AccumuloServerContext", "class2": "CloseWriteAheadLogReferences", "tool_pred": 1, "ground_truth": 1, "deepseek_pred": 0, "qwen_pred": 1, "gpt_pred": 0 } The fields are interpreted as follows: class1, class2: The two API class names being evaluated. tool_pred: The judgement of APIrel. ground_truth: The manual label. deepseek_pred, qwen_pred, gpt_pred: The normalized model decision. 1 corresponds to a Yes answer, and 0 corresponds to a No answer. Frequent API relevant calls mined from clients We used SearchCode to collect clients for our subjects. We mined frequent relevant APIs from collected clients. These frequent patterns are listed in the mined directory. Infrequent API relevant calls predicted by our trained classifier Our tool, APIRel, trains a classifier based on API documents and mined relevant APIs. Its trained classifier predicts infrequent relevant APIs that do not appear in clients or documents: accumulo: https://tohidemyname.github.io/accumuloruledoc/ cassandra: https://tohidemyname.github.io/cassandradocrule/ karaf: https://tohidemyname.github.io/karafruledoc/ lucene: https://tohidemyname.github.io/luceneruledoc/ poi: https://tohidemyname.github.io/poiruledoc/ In the above documents, we argumented both mined and our predicted patterns. In particular, if a pattern appears in mined ones or it appears in both mined and predicted ones, its sentence starts with "Often called with": https://tohidemyname.github.io/accumuloruledoc/org/apache/accumulo/core/client/BatchWriter.html If a pattern appears in only our predicted ones, its sentence starts with "Shall be called with": https://tohidemyname.github.io/accumuloruledoc/org/apache/accumulo/cluster/RemoteShell.html Notice Google can fail to search the contents from github project websites. Our predicted call set of accumulo can be retrieved through: https://github.com/tohidemyname/accumuloruledoc/search?q=shall+be+called The urls of other projects can be constructed similarly.