Searching for the identity of missing migrants using artificial intelligence

Every year, hundreds of thousands of people leave their homes and countries in search of a better life or escape from violence. Many wounded or dead are on the way. Many other people disappear without their loved ones knowing if they are alive or dead, or what happened to them.

According to the International Organization for Migration (IOM) Missing Migrants Project, we have lost track of 45,000 migrants worldwide since 2014, including 24,000 in the Mediterranean.

In 2020, the INSA (National Institute of Applied Sciences) group was approached by the Transregional Forensic Team of the International Committee of the Red Cross (ICRC), which aims to improve the identification of deceased migrants in the Euro-Mediterranean region. There have been many drownings here – 16,000 since 2014. As far as we know, this effort led by anthropologist José Pablo Paraíbar of the International Committee of the Red Cross is the only one that addresses this problem across the region.

Thus the INSA teams stepped in to propose solutions to this essential identification work of the International Committee of the Red Cross, which has to deal with a large number of cases, and scattered or poor quality information on missing persons.

After a pilot project led by INSA Lyon, which provided the ICRC with tools to manage information about recovered bodies, the partnership took shape. Join the INSA Alliance Program.

This program mobilizes students and teachers who are researchers in specific cases in which NGOs, such as Handicap International or the International Committee of the Red Cross, need scientific and technical expertise. In all, there are 37 students who, as part of their studies, have developed seven projects that combine the methods and tools of schools of engineering and the field knowledge of the ICRC.

Artificial intelligence at the service of humanity

In theory, the process of identifying the drowned could easily be started by identifying the deceased by their relatives using photographs. However, these documents are not always “viewable”: either these photos are of poor quality, or the bodies are so damaged and the photos are so painful that they prevent any official recognition.

This situation led us to explore the idea of ​​comparing photos of deceased individuals with photos of people wanted by their relatives using facial recognition technologies.

This approach was explored in particular as part of Zachary Helween’s end of studies training in 2020. His project consists of using and then evaluating the contribution of facial recognition algorithms and models to identifying the remains of people who have been found drowned.

Concretely, it involves the adaptation and use of models machine learning, which is an artificial intelligence technology that allows a program to learn, on its own, to recognize similarities and differences in data sets. By confronting it with repeated experiences, such as learning about a person’s identity, the program trains and improves its results. This work made it possible to validate the benefit of this technique in the identification of missing persons.

To implement it, we compared photos of living immigrants with photos of deceased immigrants in the hope of obtaining positive matches. For this, we have prepared a similarity index based on a matching algorithm that makes it possible to obtain possible identity scores for a person in the form of percentages.

Everything is integrated into a web application intended for clients of the International Committee of the Red Cross who are legally responsible for the identification of remains, such as forensic institutes. This app is under development and every project aims to improve it.

The results obtained are encouraging. Thanks to this software, we were able to develop a complete prototype of facial recognition applied to missing immigrants. However, in order to be able to provide really reliable indications of similarity between images of living and dead people, thousands and thousands of images must be obtained.

Illustrative video of the face resurfacing program developed by the Insa Network of the International Committee of the Red Cross (INSA/CICR).

Having set these limits, the tool developed today offers ICRC clients the possibility to direct searches by providing a list of potential matches, which certainly makes the search tedious, but humanly feasible.

Constantly improving software

At the beginning of this project, in 2020, the specifications had to be worked out. So INSA students and their mentor Charles Dossal translated the automatic or non-automated processing that should be performed on these images into technical terms: extract the face from the decor (bag, boat bottom, table, etc.), center and align the image, reduce or remove cuts, remove foam From the mouth and bring a sparkle of life to the eyes.

Taliban in 4e In the year Adam Hamidullah and Dean Trim Van programmed the algorithms we identified as most relevant to solving these various problems. Sometimes it may be necessary to copy portions of healthy skin in order to “digitally heal” wounds or insert eyes from another face when they are severely damaged. The results were encouraging, but we were also able to measure that artificial intelligence (AI) could provide more successful answers.

During the summer of 2021, Zoé Philippon and Jeong Hwan Ko viewed these horrific images with the goal of seeing more accurately what artificial intelligence could do for this mission.

Zoé Philippon’s goal was to test the limits of facial recognition algorithms based on artificial neural networks when applied to images of deceased faces, especially of African descent. These algorithms are effective on images similar to those used to calibrate them, and here are the faces of live people, mostly white and male, with a small percentage of female or African faces.

So it ran several tests, and retrained the AI ​​to be more effective at taking pictures of missing people. The results seem to indicate that these algorithms would benefit from training more specifically on the faces of the more representative population of missing persons and that this recognition deteriorates significantly when the person to be identified is dead. Access to a larger amount of data can confirm these very encouraging initial results.

digital makeup

Jeong Hwan Ko tried to improve the results of “digital makeup” by using artificial neural networks, which were also previously tested, to fill in the gaps in the images. These methods proved very effective in erasing injuries, but to repair the mouth or the eyes, it was necessary to use other neural networks capable of inserting part of one image into another.

At the moment, the programmer chooses the image to insert, but in the future it will be more efficient to allow the algorithm to search for itself in a large database, eyes, mouth, or ears in good condition very similar to those of the face to be recognized. There is still work to be done, and here again, wider access to the data will undoubtedly improve the quality of facial reconstruction.

Today, projects continue. We are always looking for data to further train machine learning software. We are also looking for corporate sponsors willing to share technology, time, and support with us.

Finally, it is worth noting that these same apps, which were developed in response to the missing migrants crisis, can also be used in other contexts such as disasters, conflicts or any situation that may lead to the anonymity of deceased persons.


This article was co-authored by Samuel Kenny, ICRC-INSA coalition coordinator.


This article is part of the “Great Science Stories in Open Access” series, published with the support of the Ministry of Higher Education, Research and Innovation. To find out more, please visit the Openscience.fr page.

Leave a Reply

Your email address will not be published.