While organizations are utilizing AI technology, there isn't a clear understanding of how it works. Here, we explore the benefits and drawbacks of the lack of transparency in AI.
Modern AI was born when rules-based software programming was no longer able to tackle the problems the computing world wanted to solve. It wasn't possible to code every condition the program had to measure, so computational experts designed machines that imitated how humans think, enabling AI to learn by itself by observing data. The approach, known as neural networking, gave rise to AI technologies like face recognition programs, cancer detection algorithms and self-driving cars.
But neural networks came with a tradeoff: We couldn't understand how the systems worked. The AI models lacked transparency. This phenomenon is known as black box AI, and it has turned out to be quite problematic.
AI is typically measured in percentages of accuracy -- i.e., to what degree the system is able to give correct answers. Depending on the task at hand, the minimum accuracy required may vary, but accuracy, even if it is 99%, cannot be the only measure of AI's value. We must also take into account a major shortcoming of AI, especially when applying AI in business: An AI model with near-perfect accuracy can be problematic. As the accuracy of the model goes up, AI's ability to explain why it arrived at a certain answer goes down, raising an issue companies must confront: the lack of AI transparency of the model and, therefore, our human capacity to trust its results.
The black box problem was acceptable to some degree in the early days of the technology but lost its merit when algorithmic bias was spotted. For example, AI that was developed to sort resumes disqualified people for certain jobs based on their race, and AI used in banking disqualified loan applicants based on their gender. The data the AI was trained on was not balanced to include sufficient data of all kinds of people, and the historical bias that lived in the human decisions were passed on to the models.
AI also showed that a near-perfect model could still make alarming mistakes. An AI model with 99% accuracy could make errors for the remaining 1%, such as classifying a stop sign as a speed limit sign.
While these are some of the most extreme cases of misclassification -- and purposely designed adversarial inputs to fool the AI model -- they still underline the fact that the algorithm has no clue or understanding of what it is doing. AI follows a pattern to arrive at the answer, and the magic is that it does so exceptionally well, beyond human power. For the same reason, unusual alterations in the pattern make the model vulnerable, and that's also why we need AI transparency -- to know how AI reaches a conclusion.
Particularly, when using AI for critical decisions, understanding the algorithm's reasoning is imperative. An AI model that is designed to detect cancer, even if it is only 1% wrong, could threaten a life. In cases like these, AI and humans need to work together, and the task becomes much easier when the AI model can explain how it reached a certain decision. Transparency in AI makes it a team player.
Read the full story on TechTarget