Testing Reliability Layers in Language Models: Insights on Accuracy

Recent testing of six reliability layers in language models has shown that these layers primarily sort outputs by trustworthiness rather than correcting errors. The study found that methods such as voting and self-correction failed to enhance accuracy, instead only withholding uncertain records. For production systems, enhancing reliability may be achieved by replacing generative processes with deterministic lookups, focusing on yield rather than raw accuracy, and addressing extraction bottlenecks before implementing additional checks.

The research, conducted by Pushpdeep Mishra and Wejdan Bagais, highlights that while the theory posited that adding layers would increase accuracy, the results indicated otherwise. The layers evaluated included checks that compared model outputs against deterministic standards, returning provable failures to the model, sampling outputs multiple times for voting, and having a secondary model assess the outputs. Ultimately, the layers sorted answers based on trust but did not significantly correct errors.

The study utilized the CADEC corpus, which contains 1,250 annotated forum posts regarding adverse drug reactions. This annotated dataset allowed for precise scoring of each layer’s performance, revealing that a supervised system like CONORM, fine-tuned on part of CADEC, significantly outperformed the zero-shot extractor used in the study.

Key findings from the evaluation include:

• Reliability layers sorted answers by trust but did not correct errors; yield, defined as the ratio of correct answers to total records, revealed limitations in accuracy.
• Deterministic code can identify incorrect answers but cannot confirm correctness; it can catch nonexistent codes but still relies on models or human input for plausible answers.
• Recognition of codes from memory was less effective than selecting from a menu, with the model generating a significant number of non-existent codes when recalling from memory.
• Four out of five model families produced consistent outputs at temperature zero, while one model showed variability across runs.
• No layer was able to recover mentions that the extractor missed, indicating that improvements came more from aligning with annotation conventions than from domain-specific adaptations.

The findings underscore that while additional verification layers can help sort outputs, they do not inherently improve the accuracy of the model’s responses. The study emphasizes the need for reliable extraction methods and deterministic approaches to enhance the overall performance of language models.

Source: InfoQ