Apple is known to use Linux on servers, but I could see a case for BSD being in the running.
The macOS userland is based on BSD so you’d get a nice little fit there. And it’s not like some common BSD is bad at doing the job of a server. I know it can do great things.
Who knows if it was ever discussed. They wouldn’t want Windows (licensing, optics, etc) and macOS isn’t tuned to be a high performance server. Linux is incredibly performant and ubiquitous and a very very safe choice.