Implementing a new AI solution can often sound uncertain and to some point risky to shareholders. Organizations fear to waste resources and time, and fear to select the wrong technology partner. Most organizations, especially for their first AI project, want to ensure that they always choose a relevant use case and business case. A best practice to guarantee a ROI on an AI project is to segment it and to start with a Proof of Concept.

In this article, we explain what a Proof of Concept is, and how to leverage a Proof of Concept to 1. minimize risks and 2. maximize Return of Investment.

What is a Proof of Concept?

The idea of a proof of concept arises from five goals and needs: 1. the need to ensure the end value is sufficient to warrant the effort and investment in the project, 2. to check if the chosen plan of workflow is correct, 3. to determine the limitations for the solution of the specific problem, 4. to make sure the solution is technically feasible and 5. to check whether the proposed solution matches expectations. For many companies, it also helps in avoiding possible technical problems in the future. To achieve these multiple goals, a Proof of Concept re-scales the full project into a pilot that includes testing specific assumptions in order to obtain confirmation that the initial idea is feasible and applicable in practice. In other words, a proof of concept aims to explore and validate how well a particular business problem can be solved by AI.

Which value can be delivered by a PoC

Minimizing risk

A PoC can help a company, at a small scale, to verify a couple of core elements and ensure it goes into the right direction, without investing too much time and resources. More specifically, with a PoC, the company can test an initial AI model on its data and know whether it already has the data it needs, or whether it should enrich its database with further internal or external data. Sometimes, even if companies hold data, they are not sure that their data hold sufficient predictive value to justify a deployment in production. Alternatively, they want to investigate which other data should be collected before deploying in production.

Generating immediate insights

When performing the scoping work for a PoC, you should never forget that you can already generate insights from the PoC stage. As counter-intuitive as it may sometimes sound, an insightful PoC can be a sufficient deliverable for many companies as it enables them to generate data-driven insights that can guide them to take immediate action. The insights can be 1. Related to the predictive value of the data, as mentioned above, or 2. Related to the particular problem you are tackling (for example : what are the combinations of events leading to churn of your customers, what are the hidden correlations in your customer base, what are the relative impact of key variables in the evolution of a material price and more).

Improve data-collection practices

By learning about these insights, companies don't only generate immediate value. They can also generate long term value as they get an opportunity to know how to better structure or collect data. This better data may be used in the future to generate new AI solutions, services and so on. In a world increasingly driven by AI, this is an important factor to take into consideration.

Getting people aligned

In some companies, a PoC has to be regarded as a first step of a change management process which consists in implementing a new kind of technology in a company. It also helps to generate buy in, and prove that "Yes, AI can work very well for our company".

Building initial bricks

In some cases, building a PoC will include to develop an initial Machine Learning model that, if the PoC is satisfactory, can be further tuned and developed in the deployment stage.

The selected use case for an initial PoC should be the outcome of this scoping stage.

Methodology for a Proof of Concept

The POC can be divided into 3 stages: preparation, development and validation.

Preparation

In the preparation stage, the main focus is defining the problem, data selection and data preparation. It is where a roadmap is designed, that describes the stages of the following proof of concept.

• Problem definition : This is the first, and most important step in the preparation stage. Machine learning problems often fail because of an insufficient definition of the problem that has to be solved, and the deliverables that the PoC should attain.

• Data Selection : Once the problem is defined, the next step in the preparation stage is data selection. In this step, we help companies explore their own data sets, determine which data is valuable with information, and if there is any missing data points in the data sets. Also, in this step it is determined which internal and external data is needed and what are the data sources which can be considered.

• Data Preparation : After selecting the data, the next step is to prepare the data. Data preparation can include sorting, structuring, pre-processing and filling the missing data points (if any). Once the data is analyzed and prepared, the next stage of the proof of concept can begin - development of the model (machine learning algorithm).

Development

The development stage is where the initial model is developed, tested and tuned. It is the core stage in proof of concept.

• Development and modelling : In the modelling stage, a Machine Learning technique will be chosen. Rather than build custom algorithms, we often recommend using existing algorithms at this stage. We advise companies to understand which algorithms are selected by the Machine Learning team, as different algorithms have different trade-offs.

• Collaboration : Our experience shows that the most successful projects have a close feedback loop between business teams and data scientists, as this allows for rapid iteration and validation of the trained models.

• Testing : Once the model is built, the next step is testing the model. Testing usually involves checking how well the model works on data it has never seen. This testing procedure allows data scientists to see how well a model generalizes, and how likely it will show a strong performance in a production setting. Furthermore, this testing can also involve checking the logical steps that an algorithm has learned, and checking whether this matches with the knowledge and intuition of business teams.

Validation

This is the final stage in the proof of concept, where we present the initial insights, evaluate the results and issue recommendations for the deployment strategy. This is a key stage where all relevant company stakeholders attend so the outputs and recommendations are discussed and validated. They are usually used by our client for their deployment roadmap, or data collection, or data governance roadmap.

Why is proof of concept important?

A Proof of Concept is one of the most important stages in implementing an AI-based solution. Usually, the process takes from a few days to a couple of weeks. We hope this article clarified your understanding of a Proof of Concept.

If you are curious in finding out more, feel free to contact us and our team will answer all your questions. You can also book a 15 mn slot to discuss your idea and have a first exchange (in French, English or Dutch).