Non-invasive prospective Pilot in a Live Environment for the Improvement of the diagnosis of skin pathologies in primary care and dermatology - LEGIT.HEALTH_SAN_2024

Study type

• Study topic:
  Disease /health condition

  Medical device
• Study topic, other:
  Improvement in the diagnosis of skin conditions in primary care and dermatology with the help of the medical device Legit.Health
• Study type:
  Non-interventional study
• Scope of the study:
  Effectiveness study (incl. comparative)

  Validation of study variables (exposure outcome covariate)
• Data collection methods:
  Combined primary data collection and secondary use of data

Non-interventional study

• Non-interventional study design:
  Cross-sectional

Study drug and medical condition

• Medical condition to be studied:
  Ulcer

  Urticaria

  Seborrhoeic keratosis

  Psoriasis

  Onychomycosis

  Malignant melanoma

  Herpes simplex

  Granuloma annulare

  Dermatitis

  Alopecia

  Acne
• Additional medical condition(s):
  Tinea; Nevus

Population studied

• Short description of the study population:
  Study Population and Dataset Description; ; The study population consisted exclusively of Healthcare Professionals (HCPs) rather than patients. A total of 16 physicians were recruited to validate the diagnostic accuracy and utility of the medical device. The cohort was stratified by speciality to assess performance across different levels of dermatological expertise, comprising 10 primary care practitioners and 6 dermatologists.; ; The inclusion criteria required participants to be board-certified primary care practitioners or dermatologists, regardless of their years of professional experience. The study did not place specific emphasis on gender, age, or nationality as primary factors for inclusion, aiming instead for a diverse cohort. There were no specific exclusion criteria for the healthcare professionals; exclusion criteria defined in the protocol applied strictly to the quality of clinical images used in the dataset.; ; Participants acted as their own control group. Each physician was tasked with evaluating a dataset of 29 validated clinical images representing a diverse range of skin pathologies. These images were previously confirmed by dermatologists and, for skin cancer cases, by anatomical pathology.; ; The composition of the 29-image dataset evaluated by each participant included the following conditions: Dermatitis (5 images), Nevus (4 images), Melanoma (3 images), Psoriasis (3 images), Alopecia (2 images), Herpes (2 images), Tinea (2 images), Onychomycosis (2 images), Acne (2 images), Urticaria (1 image), Granuloma annulare (1 image), Seborrheic keratosis (1 image), and Pressure ulcer (1 image).; ; Regarding study adherence, 12 of the 16 participants completed the entire process (reviewing all 29 images). The remaining 4 participants reviewed a partial number of images, specifically 28, 15, 9, and 1, respectively.
• Estimated number of subjects:
  16

Study design details

• Study design:
  Prospective, observational, analytical, and cross-sectional validation study. A pre-post design was used where 16 HCPs diagnosed 29 validated images first without assistance, then with AI support (Legit.Health Plus) to assess diagnostic accuracy and referral decisions.;
• Main study objective:
  To validate that the information provided by the Legit.Health Plus medical device increases the true diagnostic accuracy of healthcare professionals (HCPs) in the diagnosis of multiple dermatological conditions.; ; Secondary objectives included:; ; Validating the percentage of cases that should be referred to a specialist based on the device's information.; ; Validating the percentage of cases that could be handled remotely (teledermatology) using the device.; ; Confirming the perceived clinical utility of the device by specialists.
• Setting:
  Persons: 16 Healthcare Professionals participated, comprising 10 Primary Care Practitioners and 6 Dermatologists. Inclusion criteria required board-certified physicians regardless of professional experience. Place: The study was conducted in a remote setting via a dedicated website. Time Period: The study was conducted from June 01, 2024, to October 10, 2024. Selection: Participants evaluated a dataset of 29 validated images representing diverse skin pathologies (e.g., Melanoma, Psoriasis, Dermatitis), confirmed by dermatologists and anatomical pathology. Arms: The study utilized a single-arm, self-controlled design. Participants acted as their own control group, diagnosing images first without the device (standard clinical practice) and subsequently using the Legit.Health Plus device to confirm or revise the diagnosis.
• Comparators:
  The study did not involve an external control group. A within-subject design was employed where the physicians served as their own control. The comparator was the healthcare professional's initial unassisted diagnosis (standard clinical practice) versus their subsequent diagnosis made with the support of the Legit.Health Plus AI analysis (top 5 diagnoses with confidence levels) for the same set of images.
• Outcomes:
  Primary Outcome: Diagnostic accuracy for multiple dermatological conditions, measured as the percentage of correct diagnoses with and without the device. Secondary Outcomes:; ; Referral Necessity: Percentage of cases identified as requiring specialist referral.; ; Remote Management: Percentage of cases identified as suitable for remote handling (teledermatology).; ; Clinical Utility: User perception measured via a questionnaire (usability, confidence, and utility scores). Impact Assessment: Categorisation of the device's influence as reinforcing, improving, no impact, or negative impact.
• Data analysis plan:
  Analysis was conducted using Python (numpy, pandas) to calculate statistical measures. The primary analysis compared diagnostic accuracy percentages before and after using the device. A McNemar test was performed to evaluate the statistical significance of the difference in accuracy. Results were stratified by speciality (Primary Care vs. Dermatologist) and by pathology.; ; For secondary objectives, the percentages of cases requiring referral or suitable for remote consultation were calculated and analysed using Pearson's chi-squared test to assess associations. Clinical utility was assessed through average scores from participant questionnaires. P-values < 0.05 were considered significant.
• Summary results:
  Overall diagnostic accuracy significantly increased from 68.08% to 88.78% (p < 0.0001) with the use of the device.; ; Primary Care: Accuracy improved substantially from 62.90% to 89.92% (+27.02%).; ; Dermatologists: Accuracy improved from 76.47% to 86.93% (+10.46%).; ; Impact: The device reinforced the diagnosis in 67.83% of cases and improved it in 20.95%.; ; Referrals: 58.1% of cases were determined not to need a specialist referral.; ; Remote Management: 55.11% of cases could be handled remotely.; ; Clinical Utility: HCPs rated the utility of the data at 7.3/10 and the design/usability at 8/10.; ; Safety: No adverse events were reported.