That was exactly what happened. I can hardly read the comment now myself and its too late to edit it. At the risk of just adding noise to the thread here it is with paragraphs:
Its all about change and shared state. Humans (unlike, say, chickens or purely functional systems) cache state in their heads about the world. In fact the brain operates so slowly compared to a processor that without caching we could not operate. For example, when we catch a ball the reason we are able to predict its trajectory is not because we measure its current speed and direction and then perform some calculation and repeat for every frame. It is because we compare it with the cached (and to some extent abstracted) memory of a thousand balls we caught before, find the closest match and do a little extrapolation from it. The book 'On Intelligence' covers this. The phrase he uses to describe the human brain is a "memory prediction machine".
The problem with state of course is not the state itself, but the fact that there is more than one copy of it. In a purely functional program (or a chicken!) there is only one copy which is passed around, nothing is retained. If you put your hand over a chickens eyes they just go completely still, nothing in, nothing out. In more stateful systems the best we can do is to minimise the sharing of state i.e assign ownership of the state to certain subsystems. As humans are stateful this also works out to be the best model for them. This means that the optimum situation for quality in programming is to have one person develop all of the code. Its not always possible to do this and so the next best thing is to try to split the solution up into subsystems with well defined interfaces. Even at this we often have to assign a small team to each subsystem and we usually try to further decompose that but effectively at this point we are sharing state between people. Everybody needs to update their cache so that they all have a very similar mental model of the solution or there will be problems. They need to update their cache all the time as it is a model of a world that is being changed by the solution itself as the objective of the solution is to change the world in some way.
The problem is that when the state of the world changes its not as simple as passing that state value to one of the actors i.e. Its not like passing some simple value like an integer around the place. The truth is that when the state of the problem changes in some signifigant way then the people involved need to have time, in peace and quiet, to update their complex model in a way that is consistent with the change. They have to figure out what the new information means in the context of the problem and more importantly they need to give their brain time to come up with a solution that fits with the new state of the problem. So there are two ways a system based on shared state between people can fail. 1) Somebody fails to get notified of a state change. 2) Somebody does not spend the time to properly integrate this into their understanding. In either case that person will go around with a faulty model of the system and that is when the bugs start to happen. As far as I can see the agile guys seem to be primarly focused on 1) and neglect 2). I think you need to work on both (given that you have no option but to work on a system with many people in which you cannot completely isolate the functionality to be implemented by each person).
So what does this have to do with office space? Have an open plan office but tell people to stay at home for the morning, or tell people to stay at home every second day. i.e. Make sure they are in communication AND make sure they have time on their own to digest the communication properly. The reason I know this works is that I worked like this as a contractor for a while and it was easily the most productive period I ever had. Much more productive that working in an open plan office all the time and even slightly more productive than working on something completely alone.
Slightly OT, but I've read that humans simplify ball catching by moving in such a way as to keep the ball in the same place in their field of view. There might be more details in the papers below, but paywalled so I don't know:
Its all about change and shared state. Humans (unlike, say, chickens or purely functional systems) cache state in their heads about the world. In fact the brain operates so slowly compared to a processor that without caching we could not operate. For example, when we catch a ball the reason we are able to predict its trajectory is not because we measure its current speed and direction and then perform some calculation and repeat for every frame. It is because we compare it with the cached (and to some extent abstracted) memory of a thousand balls we caught before, find the closest match and do a little extrapolation from it. The book 'On Intelligence' covers this. The phrase he uses to describe the human brain is a "memory prediction machine".
The problem with state of course is not the state itself, but the fact that there is more than one copy of it. In a purely functional program (or a chicken!) there is only one copy which is passed around, nothing is retained. If you put your hand over a chickens eyes they just go completely still, nothing in, nothing out. In more stateful systems the best we can do is to minimise the sharing of state i.e assign ownership of the state to certain subsystems. As humans are stateful this also works out to be the best model for them. This means that the optimum situation for quality in programming is to have one person develop all of the code. Its not always possible to do this and so the next best thing is to try to split the solution up into subsystems with well defined interfaces. Even at this we often have to assign a small team to each subsystem and we usually try to further decompose that but effectively at this point we are sharing state between people. Everybody needs to update their cache so that they all have a very similar mental model of the solution or there will be problems. They need to update their cache all the time as it is a model of a world that is being changed by the solution itself as the objective of the solution is to change the world in some way.
The problem is that when the state of the world changes its not as simple as passing that state value to one of the actors i.e. Its not like passing some simple value like an integer around the place. The truth is that when the state of the problem changes in some signifigant way then the people involved need to have time, in peace and quiet, to update their complex model in a way that is consistent with the change. They have to figure out what the new information means in the context of the problem and more importantly they need to give their brain time to come up with a solution that fits with the new state of the problem. So there are two ways a system based on shared state between people can fail. 1) Somebody fails to get notified of a state change. 2) Somebody does not spend the time to properly integrate this into their understanding. In either case that person will go around with a faulty model of the system and that is when the bugs start to happen. As far as I can see the agile guys seem to be primarly focused on 1) and neglect 2). I think you need to work on both (given that you have no option but to work on a system with many people in which you cannot completely isolate the functionality to be implemented by each person).
So what does this have to do with office space? Have an open plan office but tell people to stay at home for the morning, or tell people to stay at home every second day. i.e. Make sure they are in communication AND make sure they have time on their own to digest the communication properly. The reason I know this works is that I worked like this as a contractor for a while and it was easily the most productive period I ever had. Much more productive that working in an open plan office all the time and even slightly more productive than working on something completely alone.