Isn't recursion solved by just feeding the output of the LLM (with some additional instructions) back into the input? That's how a lot of complex problems are already tackled with LLMs.
Perhaps similarly, neural activation patterns in biological neural networks are cyclic: there are cycles in the graph (which has oscillating voltage due to the heart being an electrical generator in addition to a pump).
