Syntax error!
Why syntax error? It would perfectly make sense.
Further, read the article how-would-you-like-your-loops-served-today
Then I tested with the best shell ever, zsh.
It works!
REDIRECTION
Before a command is executed, its input and output may be redirected
using a special notation interpreted by the shell. Redirection may
also be used to open and close files for the current shell execution
environment. The following redirection operators may precede or appear
anywhere within a simple command or may follow a command. Redirections
are processed in the order they appear, from left to right.
-- excerpt from man bash
Because the statement in question is not a simple command, then syntax, design, and code prevail over what may seem logical.
I note that ksh and dash also fail to process as you desire, but, I agree, zsh seems OK with the short tests I did.
while is not an external command, it's a shell builtin. The same rules don't always apply. That you can redirect into them at all is a huge blessing and feature rather unique to Bourne-based shells.
Minor quibble, not a builtin, but a reserved word:
RESERVED WORDS
Reserved words are words that have a special meaning to the shell. The
following words are recognized as reserved when unquoted and either the
first word of a simple command (see SHELL GRAMMAR below) or the third
word of a case or for command:
! case do done elif else esac fi for function if in select then until
while { } time [[ ]]