Reasoning with words
Do large language models really need words to reason? Why do we all decide that LLMs need reasoning words?
It all started with chain-of-thoughts prompting. The newborn LLMs cannot follow our instructions. They have in-context learning as an innate ability. This allows them to recognize patterns in user prompts and follow them. The era of prompt engineering.
As LLMs become more powerful, I argue that we still need chain-of-thoughts.
I want to share the latest research from Meta. The researchers are finding ways to NOT use words to reason [1]. They are using reasoning tokens and continuously feed those tokens to LLMs. Reasoning tokens here are not words. It's only understandable by LLMs although we can probe it to become explainable. Excellent youtube video here. [2]
I agree with them. Why are we forcing LLMs to reason like us? They are not humans. Their reasoning doesn't need to be like ours. Biology of LLM research by Claude AI shows that LLMs have internal reasoning mechanisms [3].
Thinking of chess playing ability by LLMs. What do you think it's easier to train? 1. Generating the best move as one token 2. Reasoning words and then a move?
Reasoning words are to keep LLMs in check. We want to know if LLMs are plotting something suspicious.
Then, allow me to propose a research direction. Let's introduce a new head to LLMs. Token generation head and reasoning head. We can only enable reasoning head when we want. That will save us inference costs in production. We can use reasoning head to train reasoning. The weights will be updated to adjust the token generation head.
Reasoning head can serve as both explainability and reasoning training. Bonus point if we can invent a differentiable reasoning mechanisms just like the attention mechanism. What do you think?