SOTAVerified

Prompt Engineering

Prompt engineering is the process of designing and refining the prompts used to generate text from language models, such as GPT-3 or similar models. The goal of prompt engineering is to improve the quality and relevance of the generated text by carefully crafting the prompts to elicit the desired responses from the model.

Prompt engineering involves several steps, including selecting the appropriate model architecture and parameters, designing the prompt format and structure, selecting the appropriate task and training data, and fine-tuning the model using the selected prompt and data.

Prompt engineering is a crucial step in the development of language models, as it can greatly influence the quality and effectiveness of the model's responses. By carefully designing and refining the prompts used to generate text, researchers and developers can improve the accuracy and relevance of the model's output, making it more useful for a wide range of applications, including chatbots, language translation, content creation, and more.

Papers

Showing 451–500 of 1236 papers

TitleStatusHype
Static Vs. Agentic Game Master AI for Facilitating Solo Role-Playing Experiences—0
Automatic Prompt Optimization via Heuristic Search: A Survey—0
Can Large Language Models Extract Customer Needs as well as Professional Analysts?—0
Representation Engineering for Large-Language Models: Survey and Research Challenges—0
A Systematic Survey of Automatic Prompt Optimization Techniques—0
Can Large Vision-Language Models Detect Images Copyright Infringement from GenAI?—0
Navigation-GPT: A Robust and Adaptive Framework Utilizing Large Language Models for Navigation Applications—0
AutoMedPrompt: A New Framework for Optimizing LLM Medical Prompts Using Textual Gradients—0
Control Illusion: The Failure of Instruction Hierarchies in Large Language ModelsCode0
Automated Query-Product Relevance Labeling using Large Language Models for E-commerce Search—0
Chats-Grid: An Iterative Retrieval Q&A Optimization Scheme Leveraging Large Model and Retrieval Enhancement Generation in smart grid—0
From Knowledge Generation to Knowledge Verification: Examining the BioMedical Generative Capabilities of ChatGPT—0
Can LLMs Predict Citation Intent? An Experimental Analysis of In-context Learning and Fine-tuning on Open LLMsCode0
Effects of Prompt Length on Domain-specific Tasks for Large Language Models—0
QUAD-LLM-MLTC: Large Language Models Ensemble Learning for Healthcare Text Multi-Label Classification—0
Exploiting Prefix-Tree in Structured Output Interfaces for Enhancing Jailbreak AttackingCode0
Navigating Semantic Relations: Challenges for Language Models in Abstract Common-Sense Reasoning—0
Personalized Education with Generative AI and Digital Twins: VR, RAG, and Zero-Shot Sentiment Analysis for Industry 4.0 Workforce Development—0
UM_FHS at TREC 2024 PLABA: Exploration of Fine-tuning and AI agent approach for plain language adaptations of biomedical text—0
RAG-Gym: Optimizing Reasoning and Search Agents with Process Supervision—0
Prompting a Weighting Mechanism into LLM-as-a-Judge in Two-Step: A Case Study—0
Testing Prompt Engineering Methods for Knowledge Extraction from TextCode0
EDGE: Efficient Data Selection for LLM Agents via Guideline Effectiveness—0
A Survey of Automatic Prompt Engineering: An Optimization Perspective—0
Exploring Large Language Models in Healthcare: Insights into Corpora Sources, Customization Strategies, and Evaluation Metrics—0
Intelligent Mobile AI-Generated Content Services via Interactive Prompt Engineering and Dynamic Service Provisioning—0
UnitCoder: Scalable Iterative Code Synthesis with Unit Test Guidance—0
ADO: Automatic Data Optimization for Inputs in LLM Prompts—0
Prompting in the Dark: Assessing Human Performance in Prompt Engineering for Data Labeling When Gold Labels Are Absent—0
PCGRLLM: Large Language Model-Driven Reward Design for Procedural Content Generation Reinforcement Learning—0
Evaluating improvements on using Large Language Models (LLMs) for property extraction in the Open Research Knowledge Graph (ORKG)Code0
The Ann Arbor Architecture for Agent-Oriented ProgrammingCode0
Has My System Prompt Been Used? Large Language Model Prompt Membership Inference—0
Optimizing GPT for Video Understanding: Zero-Shot Performance and Prompt Engineering—0
EvoFlow: Evolving Diverse Agentic Workflows On The Fly—0
Proceedings 40th International Conference on Logic Programming—0
SnipGen: A Mining Repository Framework for Evaluating LLMs for Code—0
Benchmarking Prompt Engineering Techniques for Secure Code Generation with GPT Models—0
Reformulation for Pretraining Data Augmentation—0
FACTER: Fairness-Aware Conformal Thresholding and Prompt Engineering for Enabling Fair LLM-Based Recommender Systems—0
OPTIC: Optimizing Patient-Provider Triaging & Improving Communications in Clinical Operations using GPT-4 Data Labeling and Model Distillation—0
KDA: A Knowledge-Distilled Attacker for Generating Diverse Prompts to Jailbreak LLMs—0
Automatic Prompt Optimization Techniques: Exploring the Potential for Synthetic Data Generation—0
Large Language Model as Universal Retriever in Industrial-Scale Recommender System—0
LLM-TA: An LLM-Enhanced Thematic Analysis Pipeline for Transcripts from Parents of Children with Congenital Heart DiseaseCode0
Joint Optimization of Prompt Security and System Performance in Edge-Cloud LLM Systems—0
Human Re-ID Meets LVLMs: What can we expect?—0
Leveraging LLM Agents for Automated Optimization Modeling for SASP Problems: A Graph-RAG based Approach—0
Arbitrary Data as Images: Fusion of Patient Data Across Modalities and Irregular Intervals with Vision Transformers—0
Semantic Web and Creative AI -- A Technical Report from ISWS 2023—0
Show:102550
← PrevPage 10 of 25Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1PromptKDHarmonic mean77.62—Unverified
2Customized EnsembleHarmonic mean75.49—Unverified
3MMRLHarmonic mean74.45—Unverified
4MMRL++Harmonic mean74.44—Unverified
5CoPromptHarmonic mean74.33—Unverified
6HPT++Harmonic mean74.24—Unverified
7HPTHarmonic mean74.17—Unverified
8ProMetaRHarmonic mean74.09—Unverified
9MetaPromptHarmonic mean74.02—Unverified
10DePTHarmonic mean74.02—Unverified
#ModelMetricClaimedVerifiedStatus
1PromptKDHarmonic mean97.77—Unverified
2HPT++Harmonic mean96.96—Unverified
3MMRL++Harmonic mean96.75—Unverified
4MMRLHarmonic mean96.68—Unverified
5HPTHarmonic mean96.65—Unverified
6CoPromptHarmonic mean96.55—Unverified
7MetaPromptHarmonic mean96.32—Unverified
8DePTHarmonic mean96.28—Unverified
9ProMetaRHarmonic mean96.16—Unverified
10RPOHarmonic mean96.03—Unverified
#ModelMetricClaimedVerifiedStatus
1PromptKDHarmonic mean77.94—Unverified
2MMRL++Harmonic mean74.46—Unverified
3HPT++Harmonic mean74.23—Unverified
4MMRLHarmonic mean73.82—Unverified
5CoPromptHarmonic mean72.79—Unverified
6ProMetaRHarmonic mean72.31—Unverified
7HPTHarmonic mean72.16—Unverified
8PromptSRCHarmonic mean71.75—Unverified
9DePTHarmonic mean71.09—Unverified
10RPOHarmonic mean68.61—Unverified
#ModelMetricClaimedVerifiedStatus
1MMRL++Harmonic mean91.94—Unverified
2PromptKDHarmonic mean89.14—Unverified
3HPT++Harmonic mean87.36—Unverified
4MMRLHarmonic mean87.21—Unverified
5CoPromptHarmonic mean85.84—Unverified
6ProMetaRHarmonic mean85.3—Unverified
7DePTHarmonic mean84.88—Unverified
8HPTHarmonic mean84.82—Unverified
9MetaPromptHarmonic mean83.38—Unverified
10MaPLeHarmonic mean82.35—Unverified
#ModelMetricClaimedVerifiedStatus
1PromptKDHarmonic mean45.17—Unverified
2MMRL++Harmonic mean42.24—Unverified
3HPT++Harmonic mean41.33—Unverified
4MMRLHarmonic mean41.15—Unverified
5DePTHarmonic mean40.73—Unverified
6HPTHarmonic mean40.28—Unverified
7ProMetaRHarmonic mean40.25—Unverified
8PromptSRCHarmonic mean40.15—Unverified
9CoPromptHarmonic mean39.76—Unverified
10MetaPromptHarmonic mean38.24—Unverified
#ModelMetricClaimedVerifiedStatus
1PromptKDHarmonic mean90.24—Unverified
2HPTHarmonic mean87.16—Unverified
3MMRL++Harmonic mean87.01—Unverified
4MMRLHarmonic mean86.78—Unverified
5ProMetaRHarmonic mean86.7—Unverified
6DePTHarmonic mean86.46—Unverified
7PromptSRCHarmonic mean85.95—Unverified
8HPT++Harmonic mean85.85—Unverified
9CoPromptHarmonic mean85.71—Unverified
10MetaPromptHarmonic mean84.52—Unverified
#ModelMetricClaimedVerifiedStatus
1PromptKDHarmonic mean97.15—Unverified
2HPT++Harmonic mean96.91—Unverified
3CoPromptHarmonic mean96.87—Unverified
4MMRLHarmonic mean96.74—Unverified
5HPTHarmonic mean96.71—Unverified
6MaPLeHarmonic mean96.58—Unverified
7MMRL++Harmonic mean96.51—Unverified
8ProMetaRHarmonic mean96.49—Unverified
9CoCoOpHarmonic mean96.43—Unverified
10DePTHarmonic mean96.37—Unverified
#ModelMetricClaimedVerifiedStatus
1PromptKDHarmonic mean83.13—Unverified
2MMRL++Harmonic mean78.18—Unverified
3MMRLHarmonic mean78.06—Unverified
4DePTHarmonic mean77.79—Unverified
5ProMetaRHarmonic mean76.72—Unverified
6PromptSRCHarmonic mean76.58—Unverified
7CoPromptHarmonic mean75.66—Unverified
8HPT++Harmonic mean75.59—Unverified
9HPTHarmonic mean75.57—Unverified
10MetaPromptHarmonic mean75.48—Unverified
#ModelMetricClaimedVerifiedStatus
1PromptKDHarmonic mean82.6—Unverified
2CoPromptHarmonic mean81.31—Unverified
3MMRL++Harmonic mean81.28—Unverified
4MMRLHarmonic mean81.2—Unverified
5HPT++Harmonic mean81.11—Unverified
6DePTHarmonic mean81.06—Unverified
7HPTHarmonic mean80.88—Unverified
8ProMetaRHarmonic mean80.82—Unverified
9MetaPromptHarmonic mean80.62—Unverified
10PromptSRCHarmonic mean80.52—Unverified
#ModelMetricClaimedVerifiedStatus
1PromptKDHarmonic mean86.1—Unverified
2MMRLHarmonic mean83.89—Unverified
3HPT++Harmonic mean83.81—Unverified
4MMRL++Harmonic mean83.81—Unverified
5ProMetaRHarmonic mean83.25—Unverified
6HPTHarmonic mean83.16—Unverified
7CoPromptHarmonic mean83.07—Unverified
8PromptSRCHarmonic mean82.74—Unverified
9DePTHarmonic mean82.46—Unverified
10MetaPromptHarmonic mean81.35—Unverified
#ModelMetricClaimedVerifiedStatus
1PromptKDHarmonic mean93.05—Unverified
2CoPromptHarmonic mean91.4—Unverified
3MaPLeHarmonic mean91.38—Unverified
4ProMetaRHarmonic mean91.34—Unverified
5MetaPromptHarmonic mean91.29—Unverified
6DePTHarmonic mean91.22—Unverified
7MMRL++Harmonic mean91.1—Unverified
8PromptSRCHarmonic mean91.1—Unverified
9HPT++Harmonic mean91.09—Unverified
10MMRLHarmonic mean91.03—Unverified
#ModelMetricClaimedVerifiedStatus
1POMPTop-1 accuracy %51.6—Unverified
2MMRLTop-1 accuracy %51.2—Unverified
3HPT++Top-1 accuracy %51.18—Unverified
4MaPLeTop-1 accuracy %50.9—Unverified
5PromptSRCTop-1 accuracy %50.9—Unverified
6HPTTop-1 accuracy %50.85—Unverified
7CoCoOpTop-1 accuracy %50.63—Unverified
8CoPromptTop-1 accuracy %50.5—Unverified
9CLIPTop-1 accuracy %47.77—Unverified
#ModelMetricClaimedVerifiedStatus
1POMPTop-1 accuracy %77.9—Unverified
2PromptSRCTop-1 accuracy %77.8—Unverified
3MMRLTop-1 accuracy %77.53—Unverified
4HPT++Top-1 accuracy %77.52—Unverified
5CoPromptTop-1 accuracy %77.51—Unverified
6HPTTop-1 accuracy %77.38—Unverified
7MaPLeTop-1 accuracy %76.98—Unverified
8CoCoOPTop-1 accuracy %76.18—Unverified
9CLIPTop-1 accuracy %73.96—Unverified
#ModelMetricClaimedVerifiedStatus
1POMPTop-1 accuracy %49.8—Unverified
2PromptSRCTop-1 accuracy %49.55—Unverified
3CoPromptTop-1 accuracy %49.43—Unverified
4HPTTop-1 accuracy %49.36—Unverified
5HPT++Top-1 accuracy %49.28—Unverified
6MMRLTop-1 accuracy %49.17—Unverified
7MaPLeTop-1 accuracy %49.15—Unverified
8CoCoOpTop-1 accuracy %48.75—Unverified
9CLIPTop-1 accuracy %46.15—Unverified
#ModelMetricClaimedVerifiedStatus
1HPT++Top-1 accuracy %65.31—Unverified
2HPTTop-1 accuracy %65.25—Unverified
3MMRLTop-1 accuracy %64.47—Unverified
4PromptSRCTop-1 accuracy %64.35—Unverified
5CoCoOpTop-1 accuracy %64.07—Unverified
6MaPLeTop-1 accuracy %64.07—Unverified
7POMPTop-1 accuracy %63.8—Unverified
8CLIPTop-1 accuracy %60.83—Unverified
#ModelMetricClaimedVerifiedStatus
1POMPAccuracy25.3—Unverified
2VPTAccuracy24.8—Unverified