It's not perceived as a religious holiday by most of us - of course not when majority of them are christians. For rest of us its a religious holiday. by not acknowledging it as religious event, you are just ignoring people of other faiths.

I am a person of no faith whatsoever. I celebrate Christmas. There's really not that much thought put into it.