The Design Patterns (GoF) book explicitly has a complete section about "Applicability" for every pattern it discusses, and makes no bones about circumstances where a pattern is not a good fit to a problem. Further, under "Consequences" it discusses (again: for every pattern in the book) negative as well as possible positive consequences of using the pattern. Then in "Implementation" it discusses tradeoffs, gotchas and variations...

The notion of Design Patterns was never intended to be prescriptive nor to suggest inherent "goodness". It was merely an attempt (largely successful, imho) to establish a common vocabulary for software designers, saying, effectively, "Look, lots of people faced with problem X have solved it in this way. Here are some(!) of the considerations, use-cases and tradeoffs, and here are some things you might want to think about, when using this pattern.

Not sure how the notion got so badly screwed up in peoples' minds.