To open up the AI black box and facilitate trust, companies must develop AI systems that perform reliably — that is, make correct decisions — time after time. The machine-learning models on which the systems are based must also be transparent, explainable, and able to achieve repeatable results.
Source: Rao, A. & Cameron, E. (2018). The Future of Artificial Intelligence Depends on Trust.
It still bothers me that we insist on explainability for AI systems while we’re quite happy for the decisions of clinicians to remain opaque, inaccurate, and unreliable. We need to move past the idea that there’s anything special about human intuition and that algorithms must satisfy a set of criteria that we would never dream of applying to ourselves.