In my course “Writing With And About AI” my students have been discussing benchmark testing for large language models and how companies might be “cheating on the tests” by pre-loading the test questions into their AIs. That phenomenon is called “benchmark contamination.”
If proven true for major firms that provide popular models such as Anthropic, Google, or OpenAI, it portends that progress in AI “intelligence” may be greatly exaggerated by those firms as they seek customers and investment capital.
This blog is not about AI per se, but the use of “benchmark” got me thinking of my drill press and shop bench on the farm. Careful measurements matter, so where exactly did “benchmark” come from?
Not the shop bench, as I haphazardly guessed in class. Instead, the OED entry notes that for surveyors’ tools, such “a mark takes the form of a horizontal groove cut in a surface, into which the upper surface of an angle iron would once have been inserted, forming a level surface or ‘bench’ to support a levelling staff.” If you have had the pleasure of looking at the high level of detail on British Ordinance Survey Maps, you will quickly grasp how important a good benchmark proves for careful measurements. First usage recorded is 1826 from the US, during the surveying for the Ohio and Chesapeake Canal, and the etymology remains a simple compound of two common English words.
You may use our word as noun or verb without attracting the fury of grammatical purists. If you do, just tell them “benchmark that!” and wave an angle iron at them.
Send words and metaphors my way by e-mailing me (jessid -at- richmond -dot- edu) or leaving a comment below. Want to write a guest entry? Let me know!
See all of our Metaphors of the Month here and Words of the Week here.
Image: “Mapping a feature” from Wessex Archaeology, via Flickr