Researchers have developed a new method to combat the tendency of large language models (LLMs) to repeatedly use a narrow selection of character names in generated fiction. This approach, termed "speculative naming," aims to increase name diversity without requiring model retraining or per-request adjustments.
A study published on arXiv introduces "speculative naming," a technique designed to address the repetitive nature of character names generated by large language models (LLMs). Researchers observed that LLMs often favor a limited pool of names, such as "Elara" and "Elias," leading to a predictable and unoriginal output in fictional narratives. Existing solutions to this problem are often cumbersome, requiring access to model logits, extensive retraining, or are limited to single-use prompts. The new method offers a more efficient way to control name concentration across an entire body of generated text.
The research paper, "Character-Name Concentration in LLM Fiction: Measuring Displacement and Mitigating Repetition," details an empirical study evaluating eight different strategies for controlling name concentration. The core of the proposed solution, speculative naming, adapts the "draft-verify-reject" mechanism from speculative decoding, a technique used to speed up LLM inference. Instead of applying this to individual tokens, speculative naming applies it to sets of candidate names. The method involves generating potential name sets through verbalized sampling and then scoring these sets against a "SlopTracker." This tracker uses a model-specific fingerprint to identify and penalize common or overused names, thereby encouraging greater diversity in the names chosen for fictional characters.
This issue of repetitive naming by LLMs has been noted in other contexts as well. For instance, a Forbes article highlighted how generative AI frequently reuses names like "Elena Vasquez" and "Marcus Chen," attributing this to the models' training on vast datasets that prioritize statistically probable and plausible name combinations over novelty. This tendency can lead to a situation where AI-generated content pollutes future training data, blurring the lines between human and machine-created text. The Forbes report also suggested prompt-level strategies to mitigate this, such as explicitly instructing the AI to avoid common placeholder or frequently used names.
The arXiv paper's authors aim to systematically measure how effective prompt-level strategies are in reducing name concentration and assess the trade-off in terms of name naturalness. Their introduction of speculative naming provides a fine-tuning-free method that operates on a different principle, adapting a core LLM inference technique to the specific problem of name diversity. The speculative naming approach requests candidate name sets and evaluates them against a SlopTracker, which combines a per-model fingerprint with other metrics to assess name usage. This allows for more controlled and consistent mitigation of name repetition across a corpus of generated text.
The problem of LLMs generating repetitive or generic content is not limited to character names. Research into LLM capabilities has explored various aspects of their text generation, including their ability to understand character profiles and mimic literary styles. However, the specific issue of name concentration, while a symptom of broader LLM generation patterns, has not been as systematically addressed. The Forbes article noted that LLMs are not truly choosing names randomly but are instead selecting popular choices from their training data. This means that while LLMs can generate meaningful names, they often default to the most common or plausible options, leading to the observed repetition.
The speculative naming method presented in the arXiv paper offers a novel solution by adapting an existing LLM inference optimization technique. By applying the draft-verify-reject paradigm to name generation, the researchers aim to introduce more variety without the computational cost of retraining or the limitations of single-request prompt engineering. The SlopTracker component, which uses a model-specific fingerprint, is key to identifying and penalizing overused names, thereby encouraging the LLM to explore a wider range of naming possibilities. This research contributes to a growing body of work focused on improving the quality and originality of LLM-generated content.