Years ago I watched MIT's course 6.033. It is a great way to understand the first principles at hand in distributed / non-trivial programs: http://ocw.mit.edu/courses/electrical-engineering-and-comput... (also on iTunesU).

If you're just getting started with this stuff, having a solid understanding of these principles is critical. I can't recommend these videos enough.

While this subject is full of interesting sub-topics to investigate for real-world use, Parallel Computing is a field much wider than just a DMM. Look at systems designed more than 10 years ago for better ideas about alternative computing models (all the good papers were written by the end of the 20th century). For example, if your "big data" is just computationally-intensive normal data, you might want to look into PVM/MPI or SSI. It's not as sexy, though, because you don't use it in JavaScript.

I'd love to see tools like MPI come back into mainstream. For instance, there's some work being done optimizing MPI in java for high performance computing but not nearly as much as what's being done for tools like Hadoop.

This is just what I wanted, something that explained me all this amazing world of distributed computing and networking.

If someone could suggest a good book, that'd be interesting too (I've been looking at Unix Network Programming and The TCP/IP Guide for starting with networks).

That depends on what level of book, and what you want to learn from it. On "how real distributed systems are built", there isn't a really good book available in my opinion. I liked The Datacenter as a Computer, but it's scope is fairly narrow and is more focused on global infrastructure issues than software design issues. On the math behind distributed systems, Nancy Lynch's book Distributed Algorithms (http://www.amazon.com/Distributed-Algorithms-Kaufmann-Manage...) is my favorite, but it's getting to be quite out-of-date.

Absent a book, there are a lot of great resources online. High Scalability(http://highscalability.com/) has a good amount of quality links to distributed systems content, and some editorial content of varying quality. In general, it's a good site to follow if you're interested in how distributed systems are being used in industry. I also like to follow Aphyr's blog(http://aphyr.com/), Daniel Abadi's blog(http://dbmsmusings.blogspot.com/), and Henry Robinson's blog(http://the-paper-trail.org/blog/) and Peter Bailis's blog(http://www.bailis.org/).

As a side note, the second edition of "Datacenter as a Computer" was recently released: http://www.morganclaypool.com/doi/abs/10.2200/S00516ED2V01Y2...

Thanks.

Basically I'd like to understand all this Redis, ZeroMQ, etc. stuff, message passing, queues, etc.

Not sure if there's a book that explains all this, or if I have to do a bit of research by myself.

Just started working on an outline. I am no expert, but have faith that I can research and simplify an explanation at the same time :)

https://github.com/mixu/distsysbook/issues/2

(Edit: On hold waiting for author to wake up .. see issues list)

don't overlook the link in Robinson's upper right page

http://henryr.github.io/distributed-systems-readings/

also:

http://christophermeiklejohn.com/distributed/systems/2013/07...

The Architecture of Open Source Applications books[1], while eclectic, seem pretty good from the chapters I've read. They have a few chapters about distributed systems--HDFS, Riak, and a chapter about "Scalable Web Architecture and Distributed Systems" (haven't read it).

1: http://aosabook.org/en/index.html

Distributed Systems: Principles and Paradigms by Tannenbaum is a classic textbook. It's considered pretty dry though.

I recommend strongly MIT's Distributed Systems course, both for the lectures and the labs. See http://pdos.csail.mit.edu/6.824

As a bonus, the labs (in Go) give you invaluable hands-on knowledge.

Another really good article on distributed systems along the same lines is this one: http://www.somethingsimilar.com/2013/01/14/notes-on-distribu...

KTH ID2203 Distributed Systems, Advanced Course, with Youtube video lectures http://www.ict.kth.se/courses/ID2203/video_lectures.html

By Seif Haridi, who co-authored 'Concepts, Techniques, and Models of Computer Programming' which is well regarded among functional programmers.

I was skeptical to click this but it's piqued my interest. If it continues as cogently as it began, then the author has my congratulations: it's said that if you truly understand something, you can explain it eloquently to others.