I'm just amazed by how poorly this was implemented. Software engineers expect hardware to fail. This wasn't just the work of a newb (or I hope it wasn't). This was planned; both the requirements and the failure modes. Then implemented by another team, and the quality verified by another team, even a 3rd party. There wasn't even a cutoff if it started operating outside it's expected range, surely one of those people would have noticed the shortfalls. This software should have the same level of engineering as other systems/ parts of the plane.
Every phone I've had in the last decade has come with GPS (which includes altitude), giros and even magnetic compasses. So they can work out the angle of attack without the need for additional sensors, or at the very least a backup!
Is this a wider issue with software development in general? My guess is that many businesses are moving away from the waterful model, which can often go overboard with planning (not really a problem in aerospace..). To 'Agile', which I don't think many people truly understand (especially some managers), often sidestepping the planning and documentation to software that appears to work fine, and senior management are overjoyed with the progress and become complacent.
This software wasn't even a crucial feature. It was only needed when high power is applied to the engines (like during takeoff) to stop the nose feeling too light by existing 737 pilots. This wasn't a jet-fighter, which would become unstable without the aid of computers. It's a shame 338 died because of what is effectively a 'schoolboy error'.
Every phone I've had in the last decade has come with GPS (which includes altitude), giros and even magnetic compasses. So they can work out the angle of attack without the need for additional sensors, or at the very least a backup!
Unless you’ve launched your phone into the air in a fit of rage, it doesn’t have an angle of attack. :)
Angle of attack is the angle between the wing and the oncoming air.
You might be wondering how that’s different from how high the nose is, and you’d be justified in wondering that. Every student pilot in the world gets these two very different things mixed up! But they are very different.
Next time you’re near an airport, watch the planes as they come in for landing. Try to get a vantage point from the side if you can. You’ll notice two things: (1) the airplane is descending, and (2) the nose is flat or pointed just slightly up.
It should be clear that in this situation, the angle between the oncoming air and the wing — the angle of attack — and the angle of the nose are different. Because the airplane is descending, the angle of attack is greater than the angle of the nose.
> Every phone I've had in the last decade has come with GPS (which includes altitude), giros and even magnetic compasses. So they can work out the angle of attack without the need for additional sensors, or at the very least a backup!
"Instead, engineers were able to add just a few inches to the front landing gear and shift the engines farther forward on the wing. The engines fit, but the Max sat at a slightly uneven angle when parked.
While that design solved one problem, it created another. The larger size and new location of the engines gave the Max the tendency to tilt up during certain flight maneuvers, potentially to a dangerous angle.
To compensate, Boeing engineers created the automated anti-stall system, called MCAS, that pushed the jet’s nose down if it was lifting too high. The software was intended to operate in the background so that the Max flew just like its predecessor. Boeing didn’t mention the system in its training materials for the Max.
Boeing also designed the system to rely on a single sensor — a rarity in aviation, where redundancy is common. Several former Boeing engineers who were not directly involved in the system’s design said their colleagues most likely opted for such an approach since relying on two sensors could still create issues. If one of two sensors malfunctioned, the system could struggle to know which was right.
Airbus addressed this potential problem on some of its planes by installing three or more such sensors. Former Max engineers, including one who worked on the sensors, said adding a third sensor to the Max was a nonstarter. Previous 737s, they said, had used two and managers wanted to limit changes."
> Boeing also designed the system to rely on a single sensor — a rarity in aviation, where redundancy is common.
I worked on the C-2 Greyhound aircraft in a past life. Part of an upgrade they got, only because it was an absolute requirement to fly in European airspace, was a magnetometer. The documentation on replacing and testing the new sensor, of which there was only one, was non-existent. The documentation said that the sensor could not and would never fail.
While in another country one failed. In disbelief my Chief had me read out every wire in the system even though I knew it was the sensor immediately. The bird was down for weeks until we got a new one because supply didn't exist and I had to make up a suitable plan for testing the replacement on the fly.
I wonder what kind of practices lead to a "this system can never fail" mentality. It should be assumed everything can and will fail.
This is the second time I've had occasion to post this quote here in the last week:
"The major difference between a thing that might go wrong and a thing that cannot possibly go wrong is that when a thing that cannot possibly go wrong goes wrong it usually turns out to be impossible to get at and repair." -- Douglas Adams
>It should be assumed everything can and will fail.
And anything still seemingly working to spec over a fairly conservative cycle count has already failed. Just in ingenious ways that it hasn't yet informed you of and nobody has yet discovered. You should probably swap it out for a spare and take it apart and hit it with things till you are sure. You should probably do that now while there is still time.
"Several former Boeing engineers who were not directly involved in the system’s design said their colleagues most likely opted for such an approach since relying on two sensors could still create issues. If one of two sensors malfunctioned, the system could struggle to know which was right."
Shouldn't this have been more of a red flag not to rely on a single sensor when human lives are literally at stake?
I suspect their reasoning was: It only trims down by 0.6°, what does it matter if a faulty sensor accdentally activates it. The plane is still flyable slightly out of trim and the pilots can easily correct it.
The 0.6° number was what they said immediately after the Lion Air crash.
But their reasoning was wrong.
Turns out, MCAS was actually adjusting trim down by more like 1.5° and they never considered that a faulty sensor would trigger multiple MCAS activations and move the stablizer all the way to the end of it's range.
MCAS is not an anti-stall system, it’s a way to change stick forces by trimming the horizontal stabilizer - the surface with most control authority. There’s no evidence it has ever prevented a single stall or even activated properly, but there’s evidence it has helped crash two planes.
>Every phone I've had in the last decade has come with GPS (which includes altitude), giros and even magnetic compasses. So they can work out the angle of attack without the need for additional sensors, or at the very least a backup!
You would need a sensor to measure relative wind. Angle of attack is the angle between the wing and the relative wind.
A six axis sensor could probably measure AoA in a pinch. It’s just like how a stability control system works in a car (which is measuring slip angle, which is the lateral “angle of attack” between tires and road). If you detect upwards rotation but no matching acceleration forces, you’re detaching from the air stream.
Obviously this system will drift over time, but it could more than adequate to detect a failed AoA sensor and revert to the second one.
Since angle of attack is by definition the angle between a reference line of a body and the fluid it is moving through, I fail to see how you could measure it without measuring the relative motion of the fluid.
Your suggestion would work under the assumption that there is never any vertical motion of air, but that's not the case in real life. E.g. in a downdraft your sensor would fail to detect a stall.
That’s true, but it would still be good enough to cross check the aerodynamic AoA sensors upon a disagree between the two and select the one that’s more reliable.
Past professional experience with similar sensor systems.
I also assume the AoA sensor failure was evident over time, through several phases of the flight and not momentarily.
A sensor fusion algorithm would notice the divergence over time between the IMU and the two AoA sensors, then select the one AoA sensor that was closer.
Even in the case of a momentary failure, it would take an extreme coincidence to have the readout pattern of the failed sensor seem closer to the IMU based AoA model than the working sensor.
Keep in mind I’m only talking about cross checking the AoA sensors here; the flight control algorithms are only considering the aerodynamic sensors for AoA calculation as before, but they have an independent external reference to gauge the data quality of each sensor.
It could at least provide a reference to infer if readings were within a sensible range. Outside of that range it might be best to scream bloody murder and require manual human piloting with automated assistance at producing the desired results. (E.G. If the pilot's pulling back on the stick, they PROBABLY want the airframe to go up as a result rather than down.)
Is it really necessary to measure the relative wind or is that just how it's always been done?
What would be wrong with using an accelerometer to measure/calculate velocity with respect to pitch give you a better reading of AoA? In conjunction with a second velocity source to provide redundancy?
Ah yes, the classic "software engineer reveals one easy trick to solve a complex problem domain" post. Maybe they just needed a Raspberry Pi and a sixaxis sensor board, has anybody thought of that!?
GPS info might be something you can use for sanity-checking input, but AoA can only be reliably read from an actual instrument. Air moves, you can't rely on the plane's motion.
Also, MCAS isn't active with flaps extended, so "like during takeoff" isn't correct. MCAS won't be active during takeoff and landing because it's not designed to be active at those times.
That's what I was getting at. If the nose is at an extreme angle (or reported to be) and the plane isn't climbing/ or descending - there's something wrong!
Just after takeoff while the plane is still climbing. I'm not sure that helps matters much, as the plane is off the ground and lore's the pilots into a false sense of security as plane's working ok..
Here's the bulletin Boeing issued after the first crash. It's hidden away in the bottom left and not highlighted, so it's easy to miss: "runaway stabilizer". Unfortunately, the brief is more focussed on assuring how safe the planes are, and what counter-measures to take in the event. Rather than what a runaway event looks like.
https://theaircurrent.com/aviation-safety/boeing-nearing-737...
This is a weather vane which needs to be exposed to the elements, its not something that can be inside the aircraft. It needs to be exposed to super harsh conditions to get its reading.
I don't think that's accurate or consistent enough to meet the requirements of FAR 25.207. Load factor is constantly changing in flight. You only have a constant load factor with zero turbulence and ~~straight and level flight~~ unaccelerated flight. edited Obviously I can climb at 500fpm at 1.0g, but if the rate itself is changing I'm at something other than 1.0g.
It also can't withstand failure of the selected pitot-static system, because it depends directly on it. So now you need a mechanism to detect such failures, and use a different pitot-static system, and do all of that switching while maintaining the requirements in FAR 25.207. I'm unconvinced that's feasible.
Pitot-static failure is already a big deal for much more immediate reasons than an AoA estimate. And I'm puzzled why load factor changing would be a problem? The air data computer knows what the load factor is at any given time.
To be clear: I didn't imply that you'd want to use this as the single source of AoA, only that you can use this in concert with the physical sensors to determine if the sensors are producing a valid value. You need 3 sources to be able to throw out a bad measurement, 2 AoA sensors and an airspeed-based measurement should be a perfectly valid, single-fault-tolerant, way to outvote a bad sensor.
Remember that these accidents happened at high airspeed. You'd rip the wings off the airplane before you'd stall it at those speeds. It's not a subtle failure.
That's a ridiculous statement. All sensor data are estimates.
If the IMU and airspeed sensors are good enough to feed the artificial horizon, autopilot, etc, they're surely good enough to produce an AoA estimate that's good enough to serve as a cross-check on the physical sensor.
> That's a ridiculous statement. All sensor data are estimates.
Some more reliable than others. You have no fucking clue what direction the wind is coming down on the wings based on horizon.
You have a skewed perception of what wind is. On the ground it is always horizontal because it can't go into the ground. Once you get up into the sky the dynamics are insane. Get out of your armchair.
Are you a pilot? This is the basic way that a pilot can know whether he's close to stall without anything more than an airspeed indicator.
It's basic flight physics. The lift developed by the wing is equal to CL.q.A. A is the wing area, the dynamic pressure q is basically indicated by the airspeed indicator, and CL is a known function of AoA. Lift is by definition equal to the load factor * mass.
The pilot can "know whether he's close to stall without anything more than an airspeed indicator" only if he's not in a turn (see accelerated stall).
While you are right in theory - if you knew the load factor and indicated airspeed (IAS) you could determine AoA - how would you measure IAS in practice? Normally a pitot tube would give you incorrect readings at non-zero angles of attack, and that's why airplanes use an input from an AoA sensor for correction. Which brings you back to where you started - an AoA sensor.
You are second-guessing tens (hundreds?) of thousands of engineers who thought about these problems for more than a hundred years.
(Az = vertical acceleration.) The relationship between coefficient of lift and AoA is a known function that depends on the airfoil. As long as you're below the critical angle of attack, it's an invertible function. (If you're not below the critical AoA, you've already departed controlled flight so it's too late for MCAS.)
How so? As far as I understand, it will hold to a very high degree as long as the flight conditions are stable over the time scale needed to establish the airflow pattern over the wing, which is very short.
(If the flight conditions are that unstable, a mechanical AoA vane will struggle too, since it has mechanical inertia.)
It's a ratio, so you could use a vertical accelerometer. But this isn't enough information to derive angle of attack even with airspeed. Of course, a plane can be in straight and level flight with 1.0 g loading, at 200kts, and maintain that with many different angles of attack due to weight and center of gravity differences, both of which are changing throughout the flight as fuel is consumed from tanks in different locations.
Indeed. But the system already knows what the mass is, this is needed to calculate takeoff performance, rotation speed, stall speed, maneuvering speed, etc.
CG doesn't enter into it except to second order due to horizontal stabilizer lift.
It's still not enough information to compute AOA. Let's say I'm configured for 300kts in level flight. Now I pull back on power changing nothing else. I'll end up in a decent, and eventually that will stabilize to e.g. 1000fpm at 300kts. I'm still at 1.0g, and 300kts, but I'm in a decent, obviously angle of attack is less than in level flight. So airspeed and load factor aren't enough, nor is including mass.
Horizontal stabilizer lift is a downward force and must be countered with lift from the wing to maintain level flight. It's effectively the same as adding weight. Angle of attack absolutely changes as CG changes. This is a central purpose of weight and balance computation. W&B also changes as fuel is consumed. So you'd have to take that into account.
Anyway, you can't infer angle of attack or the coefficient of lift from only airspeed, load factor, and mass. You need more information.
I'm still at 1.0g, and 300kts, but I'm in a decent, obviously angle of attack is less than in level flight.
This is a common misconception, but is not true. The angle of attack does not care if you are in level flight, climbing, or descending. As long as you are in unaccelerated flight, the AoA is identical.
It's not exactly the same, because if the descent (or climb) angle is significant, the lift only has to supply part of the weight of the airplane (the rest being supplied by drag or thrust), so the AoA will be slightly less. But that's accounted for because the load factor will also be slightly <1G.
(The edge case is if the plane is flying straight up or down. In that situation, the wings provide no lift at all and the AoA will be zero.)
Replying specifically about the stabilizer lift discussion:
What you're saying is true, but the effect of AoA is not huge. Stabilizer lift is a small fraction of main wing lift, because it has a huge leverage. The corresponding effect on aoa (effectively stall speed) is noticeable but also not huge. CG matters mostly because of its effect on controllability, not because of its effect on stall speed (although the effect is there.)
I maintain that, for the purpose of judging whether an alpha vane has failed or you're actually about to stall, you can ignore this effect when estimating AoA.
It's exactly the same as making the statement "If I'm flying at 1.5 Vs0, I know I'm not stalling (unless I'm in a very steeply banked turn or is pulling up sharply out of a dive.)
I saw you're a CFII. If a student made the above statement, would you berate him for failing to consider CG position?
These airplanes all have inertial measurement units feeding the flight instruments. On older aircraft, there would at the very least be a simple "G-meter". There are design limits on the load factor so it's important to track it to make sure it's not exceeded.
Load factor is the number of Gs the aircraft structure is supporting. In unaccelerated flight it’s 1. Straight and level, an established climb, and an established descent are all unaccelerated.
In a turn, an airplane experiences a higher load factor as a function of the bank angle. The lift from the wings is what makes the airplane turn. Since some of the lift is being diverted to the horizontal, the total lift must be increased to keep the airplane from sinking. The result is a higher load due to the acceleration in the turn.
The beginning of a climb or descent also changes load, but once established, the load returns to 1.
> Is this a wider issue with software development in general? My guess is that many businesses are moving away from the waterful model, which can often go overboard with planning (not really a problem in aerospace..). To 'Agile', which I don't think many people truly understand (especially some managers), often sidestepping the planning and documentation to software that appears to work fine, and senior management are overjoyed with the progress and become complacent.
I think this is a good point on why Agile is so dangerous to the users. Agile allows for engineering to be micromanaged and due diligence from engineers to be overlooked. (It's not something the business people care about) Don't deliver without the safety feature? Well, you're underperforming... they're going to be let go pretty quickly.
There are two crashes directly related to this feature. So, it doesn't make sense to claim it wasn't critical.
In Boeing's own hazard analysis, they rated the feature "Hazardous", which is the second most critical rating. Obviously, it should have been rated "Catastrophic". But even with their own policy, anything rated hazardous is not allowed to have single-sensor failure. By their own policy, this should have had at least two sensors. There were multiple failures that let this slip through to production.
Sorry, I meant to write crucial (similar word, quite a different meaning..)
I believe the reason for MCAS was because the had to lower and move the engines forward. So at lower power, the forward engines make the plane nose-heavy. When the engines are near full power, the lower engines push the nose up. Compounding the difference in handling compared to the older models.
Here's the safety bulletin Boeing issued after the first crash. Even when you know what you're looking for it's hard to spot: "runaway stabilizer". It does seem they downplayed how serious the failure is, but they couldn't know how common it could be. That raises another question: Other pilots must have experienced the runaway event, but managed to gain control. What happened to those reports?
There will be lessons learnt from these crashes, in a number of fields.
> I believe the reason for MCAS was because the had to lower and move the engines forward. So at lower power, the forward engines make the plane nose-heavy. When the engines are near full power, the lower engines push the nose up. Compounding the difference in handling compared to the older models.
Wrong. If the power of the engines mattered, it'd have been an input to MCAS. Moving the engines forward changed how the nacelles created lift at high angles of attack, causing a pitch up effect from the nacelles, reducing (but not eliminating) the stick forces required to maintain the high angle of attack.
Every phone I've had in the last decade has come with GPS (which includes altitude), giros and even magnetic compasses. So they can work out the angle of attack without the need for additional sensors, or at the very least a backup!
Is this a wider issue with software development in general? My guess is that many businesses are moving away from the waterful model, which can often go overboard with planning (not really a problem in aerospace..). To 'Agile', which I don't think many people truly understand (especially some managers), often sidestepping the planning and documentation to software that appears to work fine, and senior management are overjoyed with the progress and become complacent.
This software wasn't even a crucial feature. It was only needed when high power is applied to the engines (like during takeoff) to stop the nose feeling too light by existing 737 pilots. This wasn't a jet-fighter, which would become unstable without the aid of computers. It's a shame 338 died because of what is effectively a 'schoolboy error'.