Unique Auto Prompt Selection (APS) technology outperforms existing solutions

Our AI research team's paper "Automatic Prompt Selection (APS)" has been accepted at the international conference in the fields of data mining and knowledge discovery, PAKDD 2025 (Pacific-Asia Conference on Knowledge Discovery and Data Mining) (June 10-13, Sydney, Australia).

Cinnamon AI's "Automatic Prompt Selection (APS) for LLM" image

When companies use generative AI, it is important to prepare prompts that have both the "generality" to handle a variety of situations and the "specificity" to suit specific contexts and purposes. In this paper, we demonstrate a lightweight, practical, and highly accurate framework in which APS automatically selects the optimal prompt for input from text and data provided by users. APS significantly reduces the time and cost required to design prompts when using LLM in companies, and significantly improves the speed and stability of the entire business process.

APS has achieved higher performance results than existing solutions on evaluation benchmarks and datasets (GSM8K, AQuA, MMLU, and real-world datasets in the insurance industry) that measure the practicality of LLM. APS consists of three main approaches.

● Input Data Driven Clustering
We use an LLM-based prompt generator to cluster the training data and automatically generate candidate prompts for each cluster.

● Batch prompt generation for each cluster
We synthesize a dataset of input-prompt-output tuples to train a prompt evaluator that ranks prompts based on their relevance to the input.

● Learning a Lightweight Ranking Model for Prompt Selection
For each new input, the prompt evaluator scores it and selects the best prompt for the new input.

In the future, the Cinnamon AI research team will expand the research of "Automatic Prompt Selection (APS) for LLM" announced this time to "Few-shot in-context learning" that learns with a small amount of data, aiming to apply the use of LLM to a broader range of natural language processing tasks, not just Q&A. Through the results of this research, Cinnamon AI will continue to lead the forefront of AI technology and contribute to improving corporate productivity and business innovation.

■ Related pages

PAKDD 2025
https://pakdd2025.org

Cinnamon AI research team paper "Automatic prompt selection for LLM"
https://arxiv.org/abs/2404.02717

Cinnamon AI "Super RAG"
https://cinnamon.ai/ai_model/super-rag/

Cinnamon AI "Super RAG" document download
https://contents.cinnamon.ai/download/wp_superrag_dl_lp