Methodological flaws of the recent Iron Dome criticism

Recently, Israel's left-leaning Haaretz newspaper reported that several missile-defense experts have expressed doubt on the efficacy of the IDF's Iron Dome defense system. During Operation Pillar of Defense, the system was in place to defend against short-range rockets and missiles. At the time it was considered to be a huge success despite some notable failures such as this rocket which hit and did extensive damage to an apartment building in Rishon Letzion, a suburb of Tel Aviv. 

Professor Theodore Postol, who famously disproved the alleged 96% interception rate of the Patriot missile defense system during the first Gulf War, has now claimed that the "Iron Dome's intercept rate, defined as destruction of the rocket's warhead, was relatively low, perhaps as low as 5%, but could well be lower". Postol and his colleagues studied videos of the Iron Dome in action during Pillar of Defense and determined that the appearance of a successful interception was often incorrect as there should be a secondary explosion from the warhead of the rocket. The scientists also noted that for some strange reason the Iron Dome's missiles (two are sent to intercept the incoming rocket) always exploded at the same time, even though they were in different locations and targeting the same projectile. The third bit of evidence which the scientists say are indicative of a lower interception rate is that there were over 3,000 "civilian damage reports were filed for destruction caused by incoming rockets. It is impossible, claim the scientists, that the 58 rockets that weren't intercepted - the number reported by the IDF - could have caused damage on such a large scale". 

I was in Tel Aviv during Operation Pillar of Defense and like most Tel Avivians, I spent quite a bit of time that week in the bomb shelter. There were 5 Tzevah Adom (Code Red) alarms which I heard (I did not hear the Tzevah Adom which resulted in the rocket hitting the apartment building in Rishon), one a day for 3 days (11/15/12-11/17/12, and two the last day 11/18/12). According to both Hamas, the IDF, and Israeli television stations, each time the Tzevah Adom sounded at least 2 Fajr-5 rockets were sent to Tel Aviv (EDIT: This may have been an exaggeration as I now believe that only 1 rocket was sent each time). If the warheads were not successfully intercepted and destroyed by the Iron Dome as Professor Postol and his colleagues allege, where did they go? Even with a theoretical 10% intercept rate (higher than Postol's 5% but much lower than the IDF's 84%), there would have been at least 4 other rockets which the Iron Dome would have failed to intercept over Tel Aviv. There is not much open space around Tel Aviv, and I guess that the rockets could have landed in these places, but the idea that civilian areas were struck by 4 or more 175 lb warheads and no one noticed seems beyond ridiculous. While I myself did not hear every explosion of the Iron Dome missiles or Fajr rockets, friends of mine did, so I feel it is safe to assume that the Tzevah Adoms I heard were not false alarms.

From my limited understanding of mid-air explosions and high school science classes, not to mention innumerable action movies, I am under the impression that every time something explodes in mid-air, the remnants of the original object are spread of a larger area than if the object where to strike the ground. Not every piece of the object will be immediately incinerated. Similar to small meteorite fragments such as those that recently hit Central Russia, these pieces can cause significant and widespread damage. Why do these scientists assume that once the incoming rocket is intercepted there will not be debris? This article indicates the possibility of this exact scenario I am describing where the debris from an interception (and NOT the rocket warhead) caused damage on the ground. The warhead of a Fajr-5 is certainly not responsible for this as the scale of the damage would have been much higher. This seems much more likely to be the cause of such a 'high number of damage reports' than the possibility that rockets were not successfully intercepted. Theoretically, rocket fragments will be scattered over a larger area while a warhead explosion is confined to one location. Also while not as relevant to my central argument, I am curious why the scientists would conflate the damage reports from the hundreds of Qassam rockets and mortars (which the Iron Dome or any other missile defense system may not always be able to intercept due to the short 15-20 second flight time from Gaza), and the longer range Grad and Fajr rockets which were sent towards larger cities in Southern and Central Israel?

The last and perhaps most important question is why Professor Postol changed his mind from November to now and what made him do so. He was interviewed as a missile defense expert by MIT's Technology Review in an article released on November 26th, 2012, where he claimed that the Iron Dome was effective in intercepting the rockets. I understand he may have more information now than he had then, but the conviction and authority with which he previously praised the Iron Dome has been ignored in these recent news stories and not addressed by Professor Postol himself.

While the mathematical account of the scientists seems problematic from the outset, I am not as concerned about this as much as some more basic methodological issues, and why the major media sources (who gladly picked up this story), have not made any attempt to critically analyze these claims. I am by no means convinced that the Iron Dome is a permanent (or even temporary) solution, or that it's success rate is as high as the IDF has claimed, but the arguments given by Professor Postol seem highly problematic and have not convinced me in any way. The United States has spent an enormous sum of money on overhyped military projects (like the F-35) that go nowhere and I would hate for Iron Dome to be another one of these, but until further evidence is provided, I remain unconvinced.

UPDATE 1:

I stumbled upon this Voice of America article talking about several flaws of the Iron Dome system. One of them is that the interception is based upon exploding the warhead rather than the entire rocket so there will be debris from the interception. This corroborates my point, and is another potential piece of evidence against Professor Postol's theory.

UPDATE 2:

Haaretz' Uzi Rubin has posted an article asking where the nearly 500 rockets (480 to be exact) which were understood to have either landed in or been intercepted in "built-up areas" went to. Mr. Rubin takes Professor Postol's 10% claim to indicate that 48 of the 480 would have been shot down successfully, leaving 432 rockets and their warheads unaccounted for. While it is a bit crude and does not touch on some of the other issues which I mentioned in this blog, it is  encouraging to see at least one person on a reputable site questioning Professor Postol's theory.

UPDATE 3:

The BBC's Kevin Connolly has a new short (and more or less worthless) video report on the Iron Dome. Brings up the same flawed arguments (he even states that Israel claims that 90% of the rockets fired at it were intercepted, despite the incredibly obvious fact that this is not true, and was never claimed by Israel). Shoddy journalism as it takes a 5 second Google query to note that the claim of the Israelis is that 90% of the rockets it ATTEMPTED to engage were successfully intercepted. In either case, just another example of how uncritical thinking by those in positions of authority can be problematic as they perpetuate untruths.

UPDATE 4:

Uzi Rubin has posted another, more complete refutation of Professor Postol's argument. It can be accessed here. Postol's memo is also available now and can be accessed here. This memo contains no data, and no real methodological explanation.

UPDATE 5:

Postol (and his friend Richard Lloyd) have stood by their claim of the Iron Dome not working despite increasing evidence to the contrary. In another interview with MIT's Technology Review, they have reiterated their doubt as to the system's efficacy. I am guessing that they have not been in Israel and do not understand the geography of the area, because there is no other excuse for such a poor hypothesis given the available data. If all of these rockets' warheads were not detonated where are the impact craters? It defies logic.

UPDATE 6: 

Here is the new "paper" from Postol, which includes "data". He now says the rockets' warheads are small which is why there are fewer casualties. He doesn't explain how this relates to his earlier claim that there are a large number of damage reports. I am not sure how he can be taken seriously, when he fails to provide real data, and he changes the basis for the argument without changing his conclusion or even saying WHY he changed his argument. Postol also argues that the explosions from warheads that hit their target is localized, again reinforcing my original point about mid-air interceptions causing a wider debris field and therefore more opportunities for damage.

UPDATE 7: 
I stumbled upon this post with some technical specifications of the Iron Dome. Assuming that this is correct: the Iron Dome interceptors (Tamir) have a warhead of 11 kg (~24 lbs). The warheads of Fajr-5 are 175 kg according to Wikipedia, while the Hamas/PIJ version (M-75) are 175 lb according to other sources. So let us imagine a hypothetical situation where Postol is correct and these warheads are NOT exploded. Everyone in Tel Aviv (which is a fairly large city) heard an explosion, or perhaps multiple explosions, and many had their houses shake and windows rattle. Is a 20 pound warhead big enough to cause this to happen all over the city? Secondly, if everyone is hearing the 24 pound warhead and the 175 (or 375+ if wikipedia is correct) warhead remains unexploded by the Tamir, why didn't anyone hear/feel/die from the warhead with 8-15 TIMES the mass? This is a simple logical problem with Postol and Lloyd's claims, and it is simply inexplicable that folks have not questioned them. Perhaps it is because of my geographic training and spatial awareness which makes this seem a simple question, but it is shocking that these questions are ignored by so many whose job it is to be asking them. This information circles back to a rather important point which I made in my original post: Is there a difference between Qassam/Katyusha and Grad/Fajr/M302 rockets? Perhaps the Iron Dome DOES have trouble with the shorter range rockets, but if this is the case, why does Postol conflate the two vastly different scenarios? And secondarily, why does Postol conflate ICBM defense, with the type of rocket defense that Iron Dome is intended to be? The more I learn about this, the more strange Postol's argument seems.

UPDATE 8: 
The plot thickens! Uzi Rubin in 2006 (before Iron Dome existed), wrote a report about Hezbollah's rockets during the 2006 conflict. The report is available here. About halfway through the report, Exhibit B is an image of a guard-rail with damage from what is described as a 220m Anti-Personnel rocket which Rubin said is likely to be 

• "220mm rocket," range 70 km (probably Syrian-manufactured Russian "Ouragan" Multiple Launch Rocket System [MLRS])

In Postol's report from 2013 available here, he used the same image on page 6 and wrote (on the page above) that it was from a Qassam rocket. 

Qassam rockets are NOT the same as 220mm rockets, they do not have the same warhead, or range. There are many possibilities; Rubin has misidentified the source of the damage in the image (least likely since his was the original image); Postol has used the wrong description in his paper, which damages his credibility as an expert; Postol used the image knowing it was from another rocket and lied about it on purpose. I used both TinEye.com and Images.Google.com to try and identify the earliest instance of the same image, and all that I could find was links back to Rubin's original paper. This would seem to indicate that Rubin's is the original image and he either took the photo himself, or obtained it directly from someone who was at the scene. Why would Postol take someone else's image and use a misleading description? This is troubling, since so many seem to be taking Postol's account at face-value without asking the tough (or in this case fairly simple) questions. In either case, I would hope that Postol can explain his usage of this image because it is discouraging that an MIT professor would do something so underhanded.

UPDATE 9:
I am ecstatic, finally an article that understands (some of) the basics and asks good questions. Armin Rosen from Business Insider wrote this article about the damage report side of things. While not perfect, it at least asks (some of) the questions that should be asked of Postol and his followers. 

UPDATE 10: 
Israeli intelligence analyst Yossi Melman tweeted some numbers of rockets sent from Gaza. If they are correct, the Iron Dome's "success rate" would calculate to 83.5% (# intercepted/(# intercepted+ # landed in populated areas)) 584/(584+115). Rockets that are not going to land in populated areas are not worth intercepting, as the damage is likely to be negligible, so the rockets sent to "open areas" should not be considered failures of the Iron Dome. This has been a common misunderstanding by journalists from world-renowned publications as I noted in Update 3.

Rockets summary: 3361fired 584 intercepted 115 landed in populated areas 2542 in open zones 120 disintegrated in #Gaza. daily average 120
— Yossi Melman (@yossi_melman) August 5, 2014

These numbers are not great considering that the IDF has said the system is improving from previous times, but there also has to be some adjustment for the fact that the rockets have been fired in a manner intended to test the limits of the system (e.g. multiple barrages at different cities, or massive barrages sent to one city).