Towards Assured, Agile, Applied AI/ML/DI
As shown in the figure above, and as I have written about previously, AI is entering the mainstream, analogous to software engineering of a couple of decades ago. The challenges associated with successful AI/ML deployment are unique enough that they are worthy of independent consideration. The fact that they have not been addressed successfully is reflected in a spate of recent articles listing stories of AI failure, Forbes’ claim that 9 out of 10 AI projects fail, and Tech Republic’s article claiming that 96% of organizations run into problems with AI and Machine Learning projects (thanks to researcher extraordinaire Tim McGelgunn for these links).
So we decided to take stock at Quantellia recently, reviewing our own success rate. I’m happy to report that we measured it at 76% for AI projects, and a higher rate for end-to-end projects where we were responsible for not just a proof-of-concept or other element of the project but for the entire solution.
What’s the source of our success? I feel we’ve learned the long and hard way: before Quantellia I’ve led teams since the 1980’s with projects like hazardous waste recognition for the US Department of Energy, Ribosomal binding site detection for the Human Genome Project, and forensic hair analysis for the Colorado Bureau of Investigation. And since Quantellia I count 35 project all together, in AI, DI, and ML.
Every one of these projects provided substantial head bashing lessons learned, and we observed systematic patterns of errors on the part of many of our customers. So in recent months the Quantellia tech leads: Nadine Malcolm, Mark Zangari, and myself have started to pull together the systematic patterns of success into a methodology we call Agile AI. We’re still messing around with the name a bit (should it be Applied, Assured, Agile AI?) but regardless, the methodology, which has been captured in an internal best practices guide, should save years of effort and tons of risk from a standard AI project.
We’ve now taken Agile AI for a test drive on a few engagements, and we’ve received a number of rave reviews from customers who recognize that, although no method can 100% guarantee success in such a cutting-edge area, our methods in each phase of the AI lifecycle (shown below) do help to avoid some classic mistakes and can save a lot of effort. This is especially true because so much of the AI best practices wisdom comes from academia, where this field has existed for half a century. In comparison, applied projects are relatively new, and the focus is radically different: it’s not about developing the best algorithm, or the fastest, but rather about navigating the pivot not just between product and market (as with any software project), but also between model and application, and between data and model.
This follow-on post takes a deeper dive into the diagram at the top of this post, exploring new roles, challenges, and activities as AI/ML/DI become commercialized.
Read more about Agile AI here, and drop me a line if you’re interested in engaging us for a technical review, a risk/project management assessment, and/or for us to help with one or more phases of your AI/ML/DI project.
