Gotta remember that AI isn’t occasionally hallucinating and often recalling information, it’s never recalling information, it’s always hallucinating, just that we say we like some of the hallucinations, so it does those more often.
Well, not sure what you mean by hallucinating, but there are some LLMs (chatgpt3.5-turbo-instruct) that could play chess at a good amateur level (1750 elo), and by studying them, some people found parameters in the model that seemed to represent the current state of the chess board (as in, they corresponded to the game, and changing them artificially made the LLM play as if the game as always been that way).
Gotta remember that AI isn’t occasionally hallucinating and often recalling information, it’s never recalling information, it’s always hallucinating, just that we say we like some of the hallucinations, so it does those more often.
Well, not sure what you mean by hallucinating, but there are some LLMs (chatgpt3.5-turbo-instruct) that could play chess at a good amateur level (1750 elo), and by studying them, some people found parameters in the model that seemed to represent the current state of the chess board (as in, they corresponded to the game, and changing them artificially made the LLM play as if the game as always been that way).
With the exception of information you have put within the context window
https://generativeai.pub/what-is-the-context-window-ais-memory-limit-ce8a966f954a