Interesting tidbit I discovered when developing LazyReadr: nytimes doesn't ask for login if you say you're not a browser. You don't have to say you're a search engine crawler either. Just no user-agent header is enough.