Why "imagine" college without grades, when there are plenty of (healthy, well-functioning, accredited) colleges without grades out there, with decades of experience?
True, but don't those also tend to be schools with vastly different curricula and non-traditional teaching methods?
For instance, I know about Hampshire College. I know there are other examples. But if you're a traditional school, the prospect of removing grades but not changing too much else may be too difficult to fathom. They might, rightly, see grades as tied to their identity. Change one and you change the other. That may sound absurd, but I understand it especially when institutional inertia often spans tens of decades if not over a century or more.
Brown does offer grades -- A, B, C and No Credit. No Credit in a course does not appear on the transcript as a fail, it simply disappears. If you didn't get credit in the course, you didn't get credit, no penalty for trying something and bailing.
[Edit: Additionally any course may be taken Pass/Fail, so students who prefer a no-grade experience can have it.]
It's a good compromise between grades and no grades. There's still numerical feedback for those who need it (and for those whose future grad schools or employers need it) but it is in broad buckets.
It's not that hard for professors to group their students into a couple of broad categories. I always felt when I got a B or C that I knew exactly why.
As a result of the relaxed system, Brown is one of the happiest and most curious campuses among its traditional peers. The psychological difference between the students at Brown and friends who would visit was always striking. It reminds me of pg's controversial essay about the differences between hacker employees and hacker founders.
Of course, I went to Brown so keep your salt shaker handy.