ADDI Announces Results of Inaugural Data Challenge
ADDI recently hosted our first Data Challenge—the Alzheimer’s Detection Challenge—as a means to engage the research and data science community. The challenge is a key part of our aim to end Alzheimer’s disease and related dementias by promoting open science and harnessing the power of collaboration on a global scale. This challenge was aimed at developing an automated algorithm that classifies a patient’s stage of Alzheimer’s disease progression based on a common cognitive screening tool for dementia—the Clock Drawing Test (CDT). Challenge participants used derived features extracted from the decomposition of CDT images* to build and train their algorithms.
Figure 1: Representative examples of CDT images at different stages of Alzheimer’s disease progression. “Healthy” (left) an image from an individual with no obvious signs of disease; “Mild Cognitive Impairment” (center) is a patient in an early stage of decline; “Late Alzheimer’s disease” (right) exemplifies an image from a patient with severe dementia. Note: The images shown here are not actual patient data and only intended to represent the appearance of images in progressing stages of disease.
The technical solutions were assessed on a multi-class log loss metric for their ability to successfully classify data points into one of three classes representing data from individuals exhibiting no obvious signs of disease, in an early stage of cognitive decline, or with dementia. The winning solutions were able to minimize this metric on an independent test set used for evaluating the submissions. The algorithms that generated the best scores were ranked in order to identify the top performing solutions. Prizes, including cash awards, totaling over $50,000 will be distributed in the coming weeks.
To recognize the collaborative nature of the challenge and foster community engagement, community prizes awarded by a panel of judges will also be distributed shortly. The judges considered several criteria, including: 1) participation in forums to engage with or assist other participants; 2) willingness to share full notebooks of code developed to benefit the research community; and 3) the number of likes that their communal activities received from other participants.
Overall, ADDI’s first Data Science Challenge was a huge success. Over 1,200 people registered to participate in the Challenge from more than 65 countries/regions around the globe. Prizes will be distributed to 17 participants from Argentina to Zimbabwe, in recognition of the performance of their algorithms or their efforts to collaborate and support community engagement. Challenge participants were motivated to sign up for a variety of reasons; many are actively involved in work on Alzheimer’s and related dementias, some are data scientists seeking fun, new challenges, and others have deeply personal connections to Alzheimer’s disease.
One of the Community Prize recipients, a geophysicist from Argentina named Santiago, has only recently begun his Data Science journey. Not only did Santiago choose to share his entire notebook of code with the community, he also offered a very personal account of what inspired him in his notebook:
“A few years back my grandmother was diagnosed with Alzheimer’s. This disease is a slow, crippling machine. You get to see people who are bright, happy and active suddenly lose their brightness and activeness, but you still sometimes get glimpses of their happiness. You are able to recognize these moments by the look in their eyes. My grandma went through the test; it was actually the first test that she went through. Her results, as my father told me, had all of the numbers (more than only twelve) grouped up at the top right of the clock: ‘It was as if for her the time didn't pass.’ I am a very optimistic, positive person and humor helped me cope with this journey. I will try to transmit this in this notebook.”
Every participant has made a contribution to the fight against Alzheimer’s and related dementias. We thank and congratulate all of you for sharing your expertise, insights, and code with the community. Ending Alzheimer’s and related dementias will require collaboration on a global scale, and we are inspired by the level of participation and the significant cross-disciplinary interest that has been generated by this Challenge.
Sign up here to receive future announcements about ADDI Data Challenges and other funding opportunities.