In a perfect world, all A/B and Multi-variate Tests you conduct should be as expectantly observed (and ghoulishly revisited) as the one conducted last week, pictured above.
Before the test launch of SpaceX Starship, Elon Musk said there was a one-in-three chance it would not survive to a successful landing. He was wise to set expectations. It exploded upon contact with the earth.
I loved that Musk called the mission an “awesome test,” noting “We got all the data we needed.”
Spoken like a true scientist.
How do you handle and publicize your failed tests? I hope you celebrate them for the data you gathered. Put yourself in Musk’s shoes: He paid dearly for a pile of burned shards, but in return he got measurements throughout the flight that will help him rocket above the competition (sorry) to claim the profits in manned space flight he is chasing.
The only true failed experiment is the one where the data gathered is not applied to future efforts.