I don't know much Python, but I'd likely be correct in assuming there are quite a few proven http client libraries in existence.
This looks like another reason to believe that if you build a better solution, even when the space is well-covered, that you can still win adoption, so maybe, a lot of people can stop worrying about being second.
Awesome stuff. How many hours I've wasted futzing around with urllib2 and creating my own Request handlers and all that other nonsense. This lends itself nicely to working with 3rd party APIs (Facebook, Twitter) in particular.
I've noticed that the layout of the site is uncannily similar to flask's documentation (http://flask.pocoo.org/docs/), down to the "Fork me on Github" in the top right corner. Is there some sort of software producing this kind of documentation?
Requests is hosted on a site called Read the Docs, which does documentation hosting for Sphinx projects: http://readthedocs.org/
Sphinx has become the defacto documentation tool for Python projects, and a lot of other projets as well. Notably a number of PHP projects, and Varnish the web cache use it also.
The "Fork me on Github" is pretty common on a lot of sites (Github made those graphics), but that aside I agree -- I was thinking it was another Pocoo project until I saw the author.
Looks like he just co-opted the theme (Pocoo might have it available somewhere, but I couldn't find it). Nothing wrong with that, it's a great theme.
I don't have much more info than this, but it looks like they're both generated with Sphinx (http://sphinx.pocoo.org/), going by the footer from flask, and the common source (both have `<div class="sphinxsidebar">`).
Another useful Python library by Kenneth Reitz is tablib, "Python Module for Tabular Datasets in XLS, CSV, JSON, YAML, &c. ". See https://github.com/kennethreitz/tablib
I'm almost frustrated to see this. I just learned to use urllib2, and found that it took a great deal of effort to get any reasonable behavior. I wish I had known about this before. It really looks great and I will try to use it in any future work.
A coworker introduced me to Requests a couple of weeks ago. It makes life a whole lot easier.
It does highlight my biggest gripe about Python, though. I love the language, but the class library is hit-and-miss, and anything to do with HTTP is a pretty big miss. httplib, urllib, urllib2, cookielib -- yuck. I like that Requests has a nice API, but it's still built on top of pretty bad class libs underneath. (And the Requests API does expose some warts, like having to use cookielib's CookieJars.)
I'd really love to see someone's take on a clean break from the standard modules with a simple, maleable API.
If you're looking for a "clean break", Google's httplib2 [1] might be what you want. It's somewhat lower-level than Requests, but adds a bunch of advanced stuff like caching support and keep-alives. As you might guess from the name, httplib2 is more like httplib, while Requests appears to be more like urllib2.
httplib2 is part of why you should use requests: it's far more respectable as a client but not as well documented and it still takes way too much code for basic operations. I appreciate what httplib2 is trying to do, that there's a ton of hard low-level annoyances in building a modern HTTP client, but really, just use requests instead. Kenneth Reitz is very motivated and he gets the degree to which simple things should be simple whereas httplib2 feels more like an academic exercise than something people should use to build production systems[1].
Disclosure: I'm listed in the requests AUTHORS file but can claim credit for, oh, about 0.0001% of the awesomeness.
1. http://code.google.com/p/httplib2/issues/detail?id=96 is a good example: an annoying bug which affect many people, there was a fix available for months, which worked great when I applied it in a fork and pounded a couple TB of data through it, but it took over a year to make it into trunk and even longer to make it onto PyPI where any other project which required "httplib2" would get the working version.
Have been using this for a little while now. Totally awesome.
The simple example on the home page should have one more line of code that shows that you use r.content to get to the content that came back from the request. That would seem to be the most common use case.
+1 for restkit. It has a Pythonic interface, shortcuts and completely replaces urllib, urllib2 and httplib all of which I've had some beef with before. Love restkit!
Fascinating, I am really glad that "we" the new wave of programmers are not only conserned about code quality but are also deeply concerned with "usability". Everyday, as I go back and forth using old libs vs new libs, its obvious that the quality of old libs is pretty damn good but they have crappy "usability". New libs have a bit more bugs, being that they are also "newer", but they are a pleasure to use.
I wish someone would write an equivalent of this for Python's myriad of horrible process management functions. popen, popen2, popen3 etc were a nightmare, and the newer subprocess module still doesn't have an API that I'd call memorable or obvious.
Check out psutil (http://code.google.com/p/psutil/). I discovered it the other day when I needed to get a list of open ports used by an ssh client running in a subprocess, and it was as easy as this:
sshc = psutil.Process(pid)
sshc.get_connections()
It allows you to upconvert process ids, and it has its own wrapper of Popen that returns Process objects. It also seems to have well thought out windows compatibility, if that matters to you.
How do those handle SSL and user authentication? I skimmed your links but didn't see anything.
Python's urllib2 isn't that bad for the simple stuff shown in your links, but it's a mess when you have to try to log in somewhere. Seeing in the example that logging in is as straightforward as
Yeah the authentication parts of urllib2 remind me of the horrible 20-letter classes glued together in Java, just the thing I wanted to escape from by using Python. Seeing that I'm trying to make this script interface with a website that uses both cookies and http auth this library seems like a godsend.
and wondering what "from .status_codes import codes" does?
Specifically the fact that it's not "status_codes" and has a dot prepended. Is that something special in Python (guessing no) or some convention that is used? Just curious.
"When specifying what module to import you do not have to specify the absolute name of the module. When a module or package is contained within another package it is possible to make a relative import within the same top package without having to mention the package name. By using leading dots in the specified module or package after from you can specify how high to traverse up the current package hierarchy without specifying exact names. One leading dot means the current package where the module making the import exists. Two dots means up one package level. Three dots is up two levels, etc. So if you execute from . import mod from a module in the pkg package then you will end up importing pkg.mod. If you execute from ..subpkg2 import mod from within pkg.subpkg1 you will import pkg.subpkg2.mod. The specification for relative imports is contained within PEP 328."
Quickly judging by the look, I thought "hey, Armin Ronacher just released another library!", but this one seems as well thought out as if it was designed by Armin himself.
The example written in urllib2 instantly reminds me of Java and the classy query joke [http://ejohn.org/blog/classy-query/]. Just a thought: are most Java-related APIs over-engineered, and does Java have a bad influence on many API design work?
This seems to be a fantastic project. I wish more libraries were designed this way. Library's usability is just as important as performance or feature completeness.
Restlet is a good attempt at writing an api that matches the http spec well. You can find docs for a client at http://wiki.restlet.org/docs_2.1/13-restlet/21-restlet/318-r.... It's still more verbose than it might be, would be a good starting point for writing a decent api though.
From the restlet home page "Do you want to blend your web services, web sites and web clients into
unified web applications exposing and consuming RESTful web APIs?"
The link I provided was to an example of using restlet on the client side. I would agree that the api is not as human friendly as it could be but it does describe interactions in a restful which is why I suggested it would be a good starting point for writing a friendly api.
People will rightly tell you that it does not. But I must admit that this did get me to thinking about a few small changes to WebOb along with the integration of WSGIProxy, at which point there would be some real similarities.
This looks like another reason to believe that if you build a better solution, even when the space is well-covered, that you can still win adoption, so maybe, a lot of people can stop worrying about being second.