Not even GPT-4 can do it. It starts with two correct names, then just adds a bunch of names with no letter Z in them. When prompted to fix, it makes the same error.
My theory is that once it starts making a list, it falls into a different mode and forgets the filtering rule. Like mode collapse in GANs.
My theory is that once it starts making a list, it falls into a different mode and forgets the filtering rule. Like mode collapse in GANs.