Old timers (IBM, Microsoft) and newer kids (Amazon, Google) offer freely available “state-of-the-art” AI toolkits, but all too often the applications built using them sit somewhere between slightly misguided and downright dangerous. Data science has big trust issues.
This situation is comparable to the nascent software industry of the 1960s and 70s. The response of that community was to put software engineering on a professional footing – the Software Development Lifecycle1. Now is the critical time to do the same for data science, articulating and crystallizing best practice. Otherwise, history will repeat, and we’ll make the same mistakes as in previous decades.
Let’s be clear – plenty of data science leaders, planning and implementing AI and machine learning projects, far-sighted enough to understand the importance of professionalization, are working to develop their own frameworks. But building professional engineering standards is hard, as we know only too well. Our own data science framework, RAPIDE, is many years in the making, and we believe mature to the point of being a firm reference for others.