Domain Driven AI | Pedro MadrugaTable of ContentsWhy AI-based projects fail?<br>Domain Driven AI<br>How does domain knowledge impact AI development?<br>Structuring around Domain Knowledge<br>Disclaimer<br>References
Early 2023 had me obsessing about why AI-based products fail. There were so many unanswered questions back then: why do they rarely leave the proof-of-concept stage? And the ones who do, why do they fail afterwards?<br>If you were also doing Data Science & AI for some years like me, you’d quickly realize that the existing research on failed products confirmed reality: most AI projects fail. And I wanted to know why, exactly.<br>My quest for understanding the reasons behind failed projects was straightforward: 2022 saw the launch of ChatGPT, and we all saw that the age of GenAI-based products arrived.<br>And that changed everything, especially in the legal industry where the value proposition was obvious. It then became clear that the lawyers would benefit from a dramatic increase in speed when performing legal tasks, such as research. That also meant the market was about to be flooded with startups that catered to the legal industry.<br>The race for a successful product was on. And I was hunting for its formula.<br>Why AI-based projects fail?#<br>Digging through several research papers about successful data science and AI projects (written up until 2023), I confirmed that there’s a multitude of reasons for a project to die even before they’re born. Wrong processes, lack of proper data, wrong people, you name it. Nothing new here because AI products are a combination of multiple factors, that require a well-tuned orchestration in order to succeed.<br>Some factors matter more than others but, to make it even more complex, “there’s no [process] model with wide acceptance”1 - what works in one domain/project/product/company doesn’t translate to others.<br>Truth is, there is no exact formula for success. However…<br>Despite these papers presenting cases with different methodologies and processes, with different reasons to fail and succeed, there was sort of a needle in the haystack, that kept appearing over and over. It presented itself with slightly different terms but the overlapping existed and the message was clear: understand the business, especially the domain.<br>In a paper that analyzed 26 other papers, “business understanding”2 was mentioned in 68% of those, as something that can benefit teams and projects, and its absence was a major factor in a project’s failure. 3 In Martinez et al (2021) the authors include that knowing “barely some domain information” and expecting the “team will do the ‘magic’ by itself” 4 is another cause of failure. In the same paper, “describing precisely stakeholders’ needs” was the number one factor for a project success.<br>This research is no exception in the legal domain too, since it’s quite often that the stakeholders are domain experts already. Translating their needs into how the AI is developed is fundamental for better outcomes of projects and products.<br>It’s 2023, I had read the papers so by the time I was creating the first legal assistant in my company, it was time to put knowledge into practice. I had to understand what Domain Driven AI really meant in practice.<br>Domain Driven AI#<br>Domain Driven AI draws inspiration from the term “Domain Driven Design”, coined by Eric Evans in his 2003 book with the same name. Domain Driven AI follows similar principles.<br>Domain Driven AI is about gathering every possible piece of information about a given domain, and then translating that into AI development, such that it helps building or improving AI-systems and AI-based products.<br>It’s important to recognize that you, as an AI practitioner, probably don’t have knowledge of the domain you are in. If you studied subject matter and AI development, then you’re probably a unicorn. But for the rest of us who have been in different organizations and industries, we need to hunt that knowledge.<br>This is something I realized when working in the legal industry. I’m not a lawyer, I know nothing about legal systems - especially the Danish legal system. Back in 2023, I had to start from nothing. I was lucky to be part of a well-established organization like Karnov Group, one with abundance of domain experts who bore with my simple questions - and to this day they still do. It’s a catastrophic misconception that domain knowledge can be acquired within 6 months, especially in a domain like legal.<br>It’s not hard to find this knowledge. In fact, within an organization, this information is everywhere: customers, colleagues from other departments, internal and external literature, and so on.<br>It depends on factors such as size and age of the company but if you’re working in a company in a given domain, chances are that someone else knows something. And...