Machine Learning and Artificial Intelligence are for a lot of people new technologies with a lot of potentials and they want to use them inside their company. But there are several technical challenges that can impact project management, but also return on investment calculations. One of the biggest challenges comes from the data. It is well known that ML and AI require a huge quantity of data, but their quality is also essential.

Moreover, Megler (2019) warns us about biased data that discriminate certain categories. It is, therefore, necessary to be vigilant about the way in which the data are obtained and to have a critical eye on the veracity of the information retrieved. This bias can be induced by seasonality, special events, or partial data. Moreover, various authors agree on the challenges of parallelizing data and calculations and many non-trivial costs come from data storage and computational power required for AI use.

One solution proposed by Megler (2019) to address some of these challenges is to have statistical indicators to assess data quality. Pipino et al. (2002) propose the use of several objective measures:

The first method is to use simple ratios between the desired outcome and a total outcome such as the amount of error, completeness, or consistency of the data.
The second technique is based on the minimum and maximum values to determine if the values are realistic and to see the range of the data.
The last option is to use weighted averages, which require a good understanding of the importance that each variable has on the overall evaluation of a dimension.

All of these indicators are keys to identifying, investigating, and resolving sources of discrepancies. Other challenges are raised by Ransbotham et al. (2017) such as security or, according to Megler (2019), the ethics of data usage and processing. In addition, other challenges exist, but not much addressed by these authors, because they are related to very specific areas of AI such as “Deep Learning”. The major challenge of these technologies lies in the interpretability of their model. Unfortunately, this lack of understanding is often counterbalanced by very impressive results. However, it is sometimes difficult in some sectors to trust an algorithm whose logic is not understood as explained in the article by Manlhiot (2018).

Nevertheless, there is more to check before starting a new Artificial Intelligence project. After identifying a need or an opportunity to be solved, it is necessary to analyze several factors to verify that the project can be carried out in the best conditions.

II. Pre-project reflection

1. Human Capital Analysis

The first analysis concerns the company’s human capital. Depending on the project, specific skills will be required, so it is necessary to know if they are already present within the company.

If not, it is necessary to determine if the objective is to invest in training internal teams or simply to subcontract the task. To make this choice, it is wise to take into consideration the cost of the service. The final choice depends on the company’s strategy. To help you think about it, the following table shows the different solutions recommended according to each case.

2. Technology Capital Analysis

The second analysis focuses on the company’s technological maturity. The purpose of this step is to identify the technologies that are needed to carry out the project and to find out whether qualified people have mastered them. In addition, is it also necessary to study whether the company’s IT infrastructure is adapted to the use of these technologies?

Determining these answers is essential, because, in order to develop and industrialize a solution, it is sometimes necessary to have a large amount of computing power and servers to run the AI algorithms. It is possible to use “Infrastructure as a Service” (IaaS) or “Function as a Service” (FaaS) platforms that can handle these issues.

However, in order to make this choice, it is necessary to know the company’s policy regarding data governance. Some companies, because of their particular status, are obliged to use their own servers for confidentiality reasons. Nevertheless, even if these cloud solutions are easier to integrate, it requires special skills to configure them.

3. Data Analysis

The third step of the thinking process focuses on the analysis of the development of the algorithm. The question is whether the company wants to build the program itself, use open source code, or buy the solution offered by another company.

To better understand the answer, several variables must be taken into account. On the one hand, the development cost can be estimated by considering the skills of the teams and the estimated design time. On the other hand, the long-term technological objective must also be considered.

4. Project Goal Analysis

This long-term reflection on the objectives leads to a fourth reflection on the interest of the project. Sometimes, the objective is to increase the skills of the teams, which translates into a long-term investment. It is therefore important to make this clear from the start because learning does not necessarily imply an instant financial return. Even if this is not the primary goal, it is advisable to carry out this type of project with a business case that allows for a low positive ROI at best.

5. Analysis of Available Data

The reflection which is probably the most important is based on the study of the data. The goal here is to verify that the project team has understood and identified the different information needed. For all the people that I have interviewed, it is essential to verify the quality of the data.

First of all, is the data present within the company? If not, it is necessary to determine how to access the data and to identify the cost of obtaining it. In addition, a sufficient quantity of data is essential, as learning algorithms need a large volume to function. The whole point of this approach is to create a reliable and consistent acquisition strategy from project development to production.

However, it is sometimes also possible to use already pre-trained algorithms. Furthermore, transfer learning allows the learning to be passed on from one algorithm to another. The great advantage of these methods is that they require less data to use. However, in the experience of some companies, because the AI is not specifically designed to answer their problem, the results are sometimes disappointing.

6. Project Management Structure Analysis

Companies need to be sure that the project team structure facilitates its creation and management. Digital transformation projects require an appropriate team with clear sponsorship from a senior person in the hierarchy that allows defending the project even in case of problems. The goal is to keep the project as a priority and to facilitate the allocation of the budget for its development. According to the different interviews conducted, this step is necessary to ensure the project's sustainability and success.

7. Understanding Company Strategy

Finally, the last reflection is focused on the identification of the different criteria that respond to the company’s strategy. In addition to the financial aspect, some decision-makers consider the negative impact that the project may have on societal or ecological issues. Similarly, the impact on the company’s image is also taken into account. In the end, these aspects around the project can call into question the creation of the project. The values of companies are very important and should not be forgotten, as they can be a factor blocking the development of these technologies.

III. Conclusion

These different pre-project reflections can be found in the method described by Megler (2019). Adaptive team building and thinking around data are part of this research paper. Ransbotham et al. (2017) and Canhoto and Clear (2020) had explained that companies outsourcing without necessarily explaining why. With this analysis, it is now easier to understand the issues behind this decision. These different reflections allow us to minimize the different elements blocking the integration of AI before the project.

This complements the method proposed by Ransbotham et al. (2017) consisting of listing the risks during the project to mitigate them if necessary. However, the method proposed here anticipates difficulties from the beginning of the project. Indeed, many of the interviewees state that it is essential to manage all these issues from the start rather than during the project. In particular, the sources of financing must be clearly defined, as they block most companies in their development.

Canhoto, A. I., & Clear, F. (2020). Artificial intelligence and machine learning as business tools : A framework for diagnosing value destruction potential. Business Horizons, 63(2), 183‑193. https://doi.org/10.1016/j.bushor.2019.11.003

L’Heureux, A., Grolinger, K., Elyamany, H. F., & Capretz, M. A. M. (2017). Machine Learning With Big Data : Challenges and Approaches. IEEE Access, 5, 7776‑7797. https://doi.org/10.1109/ACCESS.2017.2696365

Manlhiot, C. (2018). Machine learning for predictive analytics in medicine : Real opportunity or overblown hype? European Heart Journal — Cardiovascular Imaging, 19(7), 727‑728. https://doi.org/10.1093/ehjci/jey041

Megler, V. M. (2019). Managing Machine Learning Projects Balance Potential with the Need for Guardrails. Amazon White Paper, February.

Pipino, L. L., Lee, Y. W., & Wang, R. Y. (2002). Data quality assessment. Communications of the ACM, 45(4), 211‑218. https://doi.org/10.1145/505248.506010

Ransbotham, S., Kiron, D., Gerbert, Ph., & Reeves, M. (2017). Reshaping Business With Artificial Intelligence : Closing the Gap Between Ambition and Action. MIT Sloan Mangement Review and The Boston Consulting Group, 59(1), 1‑17.

Zhou, L., Pan, S., Wang, J., & Vasilakos, A. V. (2017). Machine learning on big data : Opportunities and challenges. Neurocomputing, 237, 350‑361. https://doi.org/10.1016/j.neucom.2017.01.026

I. Challenges Related to Artificial Intelligence