Exactly the concept why you don't want to let whatever dashboards/alerts/etc you maintain on your systems have a "normal amount of reds/fails/spurious texts".
At some point you become immune.
It's a lot harder to notice theres 4 red lights today than the usual 2-3 vs noticing 1 when there are normally exactly 0.
Yes. The causative issue is the way in which projects are managed. Employees have no ownership of the project. If employees had ownership over which changes they think are best, a good employee would act on bringing the alerts back to zero before they take on new features or a new project. There are some obstacles:
1. Employees not having a say in which issues to work on. This pretty much leads to the death of a project in the medium term due to near-total disregard of maintenance issues and alerts.
2. Big-team ownership of a project. When everyone is in charge, no one is. This is why I advocate for a team size of exactly two for each corporate project.
3. Employees being unreasonably pressured for time. Perhaps the right framing for employees to think about it is: "If it were their own business or product, how would they do it?" This framing, combined with the backlog, should automatically help avoid spending more time than is necessary on an issue.
Not making an ethical/moral judgement here, just a practical one - is there any reason to believe that giving employees ownership of the projects will be any better than having "management" own it if all factors were truly considered ?
If every decision an employee made on features/issues/quality/time was accompanied by how much their pay was affected, would the outcomes really be better ?
The team could decide to fix all bugs before taking on a new feature, or that the 2 month allotment to a feature should really be three months to do it "right" without having to work nights/weekends, would the team really decide to do that if their paycheck was reduced by 10%, or delayed for that extra month for those new features were delivered ?
If all factors were included in the employee decision process, including the real world effect of revenue/profit on individual compensation from those decisions, it is not clear to me that employees would make any "better" decisions.
I would think that employees could be even more "short sighted" than senior management, as senior management likely has more at stake in terms of company reputation/equity/career than an employee who can change jobs easier, and an employee might choose not to "get those alerts to zero" if it meant they would have more immediate cash in their pocket.
And how would disagreements between team members be worked out if some were willing to forgo compensation to "do it right', and others wanted to cut even more corners ?
Truly having ownership means you have also financial risk.
> is there any reason to believe that giving employees ownership of the projects will be any better than having "management" own it
Non-technical management's skill level is almost always overrated. They're almost never qualified for it. Ultimately it still is management's decision, and always will be. If however management believes that employees are incapable of serving users, then it's management's fault for assigning mismatched employees.
> how much their pay was affected
Bringing pay into this discussion is a nonsensical distraction. If an employer misses two consecutive paychecks by even 1%, that's enough reason to stop showing up for work, and potentially to sue for severance+damages, and also claim unemployment wages. There is no room for any variation here.
> Truly having ownership
It should be obvious that ownership here refers to the ownership of the technical direction, not literal ownership in the way I own a backpack that I bring to work. If true financial ownership existed, the employee would be receiving substantial equity with a real tradable market value, with the risk of losing some of this equity if they were to lose their job.
> how would disagreements between team members be worked out
As noted, there would be just two employees per project, and this ought to minimize disagreements. If disagreements still exist, this is where management can assist with direction. There should always remain room for conducting diverse experiments without having to worry about which outcomes get discarded and which get used.
---
In summary, if the suggested approach is not working, it's probably because there is significant unavoidable technical debt or the employees are mismatched to the task.
> Not making an ethical/moral judgement here, just a practical one - is there any reason to believe that giving employees ownership of the projects will be any better than having "management" own it if all factors were truly considered ?
It's not either-or, the ownership is shared. As responsibility goes, the buck ultimately stops with management, but when the people in the trenches can make more of their own decisions, they'll take more pride in their work and invest accordingly in quality. Of course some managers become entirely superfluous when a team self-manages to this extent, and will fight tooth and nail to defend their fiefdom. Can't blame them, it's perfectly rational to try to keep one's job.
As for tying the quality to pay in such an immediate way, I guess it depends on who's measuring what and why. Something about metrics becoming meaningless when made into a target, I believe it's called Cunningham's Law. I have big doubts as to whether it could work effectively in any large corpo shop, they're just not built for bottom-up organization.
Been all of an engineer, a manager, and a founder/CEO, and I enjoy analyzing organizational dysfunction.
The difference between an engineer and a manager's perspective usually comes down to their job description. An engineer is hired to get the engineering right; the reason the company pays them is for their ability to marry reality to organizational goals. The reason the company hires a manager is to set those organizational goals and ensure that everybody is marching toward them. This split is explicit for a reason: it ensures that when disagreements arise, they are explicitly negotiated. Most people are bad at making complex tradeoffs, and when they have to do so, their execution velocity suffers. Indeed, the job description for someone who is hired to make complex tradeoffs is called "executive", and they purposefully have to do no real work so that their decision-making functions only in terms of cost estimates that management bubbles up, not the personal pain that will result from those decisions.
Dysfunction arises from a few major sources:
1. There's a power imbalance between management and engineering. An engineer usually only has one project; if it fails, it often means their job, even if the outcome reality dictates is that it should fail. That gives them a strong incentive to send good news up the chain even if the project is going to fail. Good management gets around this by never penalizing bad news or good-faith project failure, but good management is actually really counterintuitive, because your natural reaction is to react to negative news with negative emotions.
2. Information is lost with every explicit communication up the chain. The information an engineer provides to management is a summary of the actual state of reality; if they passed along everything, it'd require that management become an engineer. Likewise recursively along the management chain. It's not always possible to predict which information is critical to an executive's decision, and so sometimes this gets lost as the management chain plays telephone.
3. Executives and policy-makers, by definition, are the least reality-informed people in the system, but they have the final say on all the decisions. They naturally tend to overweight the things that they are informed on, like "Will we lose the contract?" or "Will we miss earnings this quarter?"
All that said, the fact that most companies have a corporate hierarchy and they largely outcompete employee-owned or founder-owned cooperatives in the marketplace tends to suggest that even with the pitfalls, this is a more efficient system. The velocity penalty from having to both make the complex decisions and execute on them outweighs all the information loss. I experienced this with my startup: the failure mode was that I'd emotionally second-guess my executive decisions, which meant that I executed slowly on them, which meant that I didn't get enough iterations or enough feedback from the market to find product/market fit. This is also why startups that do succeed tend to be ones where the idea is obvious (to the founder at least, but not necessarily to the general public). They don't need to spend much time on complex positioning decisions, and can spend that time executing, and then eventually grow the company within the niche they know well.
> All that said, the fact that most companies have a corporate hierarchy and they largely outcompete employee-owned or founder-owned cooperatives in the marketplace tends to suggest that even with the pitfalls, this is a more efficient system.
This conclusion seems nonsensical. The assumption that what's popular in thearket is popular because it's effective has only limited basis in reality. Heirarchical structures appear because power is naturally consolidating and most people have an extreme unwillingness to release power even when presented with evidence that it would improve their quality of life. It is true that employee owned companies are less effective at extracting wealth from the economy, but in my experience working for both traditional and employee owned companies, the reason is employees care more deeply about the cause. They tend to be much more efficient at providing value to the customer and paying employees better. The only people who lose out are the executives themselves which is why employee owned companies only exist when run by leaders with passion for creating value over collecting money. And that's just a rare breed.
You've touched on the reason why hierarchical corporations outcompete employee-owned-cooperatives:
> Hierarchical structures appear because power is naturally consolidating and most people have an extreme unwillingness to release power even when presented with evidence that it would improve their quality of life.
Yes, and that is a fact of human nature. Moreover, many people are happy to work in a power structure if it means that they get more money to have more power over their own life than they otherwise would. The employees are all consenting actors here too: they have the option of quitting and going to an employee-owned cooperative, but most do not, because they make a lot more money in the corporate giant. (If they did all go to the employee-owned cooperative, it would drive down wages even further, since there is a finite amount of dollars coming into their market but that would be split across more employees.)
Remember the yardstick here. Capitalism optimizes for quantity of dollars transacted. The only quality that counts is the baseline quality needed to make the transaction happen. It's probably true that people who care about the cause deliver better service - but most customers don't care enough about the service or the cause for this to translate into more dollars.
As an employee and customer, you're also free to set your own value system. And most people are happier in work that is mission- & values-aligned; my wife has certainly made that tradeoff, and at various times in my life, I have too. But there's a financial penalty for it, because lots of people want to work in places that are mission-aligned but there's only a limited amount of dollars flowing into that work, so competition for those positions drives down wages.
> most customers don't care enough about the service or the cause for this to translate into more dollars.
This is an important point as it reinforces the hierarchical structure. In an economy composed of these hierarchies, a customer is often themselves buying in service of another hierarchy and will not themselves be the end user. This reduces the demand for mission-focused work in the economy, instead reinforcing the predominance of profit-focused hierarchies.
There is a Chinese saying you can conquer a kingdom on horseback but you cannot rule it on horseback. What that means is, yes, entrepreneurial velocity and time to market predominate in startups. But if they don’t implement governance and due process, they will eventually lose what market share they gained. Left uncontrolled, internal factions and self serving behavior destroys all organisations from within.
This is a wonderful summary, very informative. Thank you. Is there a book or other source you’d recommend on the subject of organizational roles and/or dysfunction?…ideally one written with similar clarity.
One thing stood out to me:
You note that executives are the least reality-informed and are insulated from having their decisions affect personal pain. While somewhat obvious, it also seems counterintuitive in light of the usual pay structure of these hierarchies and the usual rationale for that structure. That is, they are nearly always the highest paid actors and usually have the most to gain from company success; the reasoning often being that the pay compensates for the stress of, criticality of, or experience required for their roles. Judgments aside and ignoring the role of power (which is not at all insignificant, as already mentioned by a sibling commenter), how would you account for this?
Most of these organizational theories I've developed myself from observing how actual corporate hierarchies function and trying to put myself (and sometimes actually doing it!) in each of the different roles and think about how I would act with those incentives. I did have a good grounding of Drucker and other business books early in my career, and two blog series' that have influenced my thinking are a16z's "Ones and Twos" [1] and Ribbonfarm's "Gervais principle" [2].
For executive pay, the most crucial factor is the desire to align interests between shareholders and top executive management. The whole point of having someone else manage your company is so that you don't have to think about it; this only works when the CEO, on their own initiative, will take actions that benefit you. The natural inclination of most people (and certainly most people with enough EQ to lead others) is to be loyal to the people you work with; these are the folks you see day in and day out, and your power base besides. So boards need to pay enough to make the CEO loyal to their stock package rather than the people they work with, so that when it comes time to make tough decisions like layoffs or reorgs or exec departures, they prioritize the shareholders over the people they work with.
This is also why exec packages are weighted so heavily toward stock. Most CEOs don't actually make a huge salary; median cash compensation for a CEO is about $250K [3], less than a line manager at a FANG. Median total comp is $2M (and it goes up rapidly for bigger companies), so CEOs make ~90%+ of their comp in stock, again to align incentives with shareholders.
And it's why exec searches are so difficult, and why not just anyone can fill the role (which again serves to keep compensation high). The board is looking for someone whose natural personality, values, and worldview exemplifies what the company needs right now, so that they just naturally do what the board (and shareholders) want. After all, the whole point is that the board does not want to manage the CEO; that is why you have a CEO.
There are some secondary considerations as well, like:
1.) It's good for executives to be financially independent, because you don't want fear of being unable to put food on the table to cloud their judgment. Same reason that founder cash-outs exist. If the right move for a CEO is to eliminate their position and put themselves out of a job, they should do it - but they usually control information flow to the board, so it's not always clear that a board will be able to fire them if that's the case. This is not as important for a line worker since if the right move is to eliminate their position and put themselves out of a job, there's an executive somewhere to lay them off.
2.) There's often a risk-compensation premium in an exec's demands, because you get thrown out of a job oftentimes because of things entirely beyond your control, and it can take a long time to find an equivalent exec position (very few execs get hired, after all), and if you're in a big company your reputation might be shot after a few quarters of poor business performance. Same reason why execs are often offered garden leave to find their next position after being removed from their exec role (among others like preventing theft of trade secrets and avoiding public spats between parties). So if you're smart and aren't already financially independent, you'll negotiate a package to make yourself financially independent once your stocks vest.
3.) Execs very often get their demands met, because of the earlier point about exec searches being very difficult and boards looking for the unicorn who naturally does what the organization needs. Once you find a suitable candidate, you don't want to fail to get them because you didn't offer enough, so boards tend to err on the side of paying too much rather than too little.
Another thing to note is that execs may seem overpaid relative to labor, but they are not overpaid relative to owners. A top-notch hired CEO like Andy Grove got about 1-1.5% of Intel as his compensation; meanwhile, Bob Noyce and Gordon Moore got double-digit percentages, for doing a lot less work. Sundar Pichai gets $226M/year, but relative to Alphabet's market cap, this is only 0.01%. Meanwhile, Larry Page and Sergey Brin each own about 10%. PG&E's CEO makes about $17M/year, but this is only 0.03% of the company's market cap.
There's a whole other essay to write about why owners might prefer to pay a CEO more to cut worker's wages vs. just pay the workers more, but it can basically be summed up as "there's one CEO and tens of thousands of workers, so any money you pay the CEO is dwarfed by any delta in compensation changes to the average worker. Get the CEO to cut wages and he will have saved many multiples his comp package."
What I see is a movement where line employees have a say on who is retained at the director and VP level.
The CEO reports to the board. But his immediate and second tier reports are also judged by the employees. The thought is that will give them pause before they embark on their next my way or the highway decision making. The most egregious directors who push out line employees in favor of their cronies will be fired under this evaluation.
> If employees had ownership over which changes they think are best, a good employee would act on bringing the alerts back to zero before they take on new features or a new project.
You say this but as someone who's run a large platform organization that hasn't been my experience. Sure some employees, maybe you, care about things like bringing alerts back to zero but a large number are indifferent and a small number are outright dismissive.
This is informed not just by individual personality but also by culture.
Not too long ago I pointed out a bug in someone's code who I was reviewing and instead of fixing it they said, "Oh okay, I'll look out for bugs like that when I write code in the future" then proceeded to merge and deploy their unchanged code. And in that case I'm their manager not a peer or someone from another team, they have all the incentive in the world to stop and fix the problem. It was purely a cultural thing where in their mind their code worked 'good enough' so why not deploy it and just take the feedback as something that could be done better next time.
With regard to alerts, I have written software that daytrades stocks, making a lot of trades over a lot of stocks. Let me assure you that not a single alert goes ignored, and if someone said it's okay to ignore said alerts, or to have persistent alerts that require no action, they would be losing money because in time, they will inevitably ignore a critical error. I stand by my claim that it's what sets apart good employees from those that don't care if the business lives or dies. I think a role of management is to ensure that employees understand the potential consequences to the business of the code being wrong.
Yes, there was a recent story about (yet another) Citi "fat finger" trade.
The headlines mentioned things like "the trader ignored 700 error messages to put in the trade", but listening to a podcast about it.. its more like awful systems that are always half broken is what ultimately lead to it.
The real punchline was this - the trader confused a field for entering shares quantity for notional quantity, but due to some European markets being closed, the system had a weird fallback logic that it sets the value of shares to $1, so the confirmation back to the trader was.. the correct number of dollars he expected.
So awful system designs lead to useless and numerous alerts, false confirmations, and ultimately huge errors.
> If employees had ownership over which changes they think are best, a good employee would act on bringing the alerts back to zero before they take on new features or a new project
That requires that you have good employees, which can be as rare as good management.
The more pernicious form of this, in my experience, are ignored compiler/linter/test warnings. Many codebases have a tremendous number of these warnings, devs learn to ignore them, and this important signal of code quality is effectively lost.
It's almost always worth spending the time to either fix all warnings or, after determining it's a false positive, suppressing it with a #pragma.
Once things are relatively clean, it's easy to see if new code/changes trip a warning. Often unexpected warnings are a sign of subtle bugs or at least use of undefined behaviors. Sorting those out when they come up is a heck of a lot easier than tracing a bug report back to the same warming.
In both Challenger and Columbia disasters, people noticed there might be a problem, tried to escalate it to get it fixed and failed to stop the launch, leading to disasters.
Do we know how many times people noticed a problem, it launched anyway and everything was fine?
https://m.youtube.com/watch?v=Ljzj9Msli5o&pp=ygUZbm9ybWFsaXp...