Make a language model write only text a regular expression accepts: compile the regex to a DFA, index which tokens each state allows, and mask every other token before choosing.
This interactive visualization needs JavaScript.