Non-Invasive Prospective Pilot in a Live Environment for the improvement of the diagnosis of Generalised Pustular Psoriasis

Study type

• Study topic:
  Disease /health condition

  Medical device
• Study topic, other:
  Improvement in the diagnosis of skin conditions, specially generalised pustular psoriasis, in primary care and dermatology with the help of the medical device Legit.Health
• Study type:
  Non-interventional study
• Scope of the study:
  Effectiveness study (incl. comparative)

  Validation of study variables (exposure outcome covariate)
• Data collection methods:
  Primary data collection

Non-interventional study

• Non-interventional study design:
  Cross-sectional

Study drug and medical condition

• Medical condition to be studied:
  Generalised pustular psoriasis

  Acute generalised exanthematous pustulosis

  Acne

  Acne conglobata

  Seborrhoeic keratosis

  Seborrhoeic dermatitis

  Palmoplantar pustulosis

  Psoriasis

  Pemphigus

  Hidradenitis

  Subcorneal pustular dermatosis
• Additional medical condition(s):
  Tinea corporis

Population studied

• Short description of the study population:
  Study Population and Dataset Description; ; The study population consisted exclusively of Healthcare Professionals (HCPs) rather than patients. A total of 15 physicians were recruited to validate the diagnostic accuracy of the medical device. The cohort was stratified by specialty to assess performance across different levels of dermatological expertise, comprising 11 primary care physicians and 4 dermatologists.; ; The inclusion criteria required participants to be board-certified primary care physicians or dermatologists, regardless of their years of professional experience. This criteria aimed to collect diverse professional insights and ensure the findings were generalizable across different clinical practices. The study did not place specific emphasis on gender, age, or nationality as primary factors for inclusion.; ; There were no specific exclusion criteria for the healthcare professionals. Exclusion criteria defined in the protocol applied strictly to the clinical images used in the dataset (e.g., excluding low-quality images or non-relevant pathologies) rather than the study subjects themselves.; ; Participants acted as their own control group, evaluating a dataset of 100 validated clinical images. The composition of illnesses in these images included the primary target conditions: Generalised Pustular Psoriasis (10 images) and Hidradenitis Suppurativa (10 images). To simulate real-world diagnostic challenges, the dataset also included confounders and other dermatological conditions: Eczematous dermatitis (10), Acne conglobata (10), Impetigo (10), and 5 images each of Acute Generalised Exanthematous Pustulosis, Acne, Severe inflammatory acne, Seborrheic keratosis, Seborrheic dermatitis, Palmoplantar pustulosis, Plaque psoriasis, Pemphigus vulgaris, Subcorneal pustular dermatosis, and Tinea corporis.; ; Regarding study adherence, 9 of the 15 participants completed the full review of 100 images, while the remaining 6 participants reviewed a partial number of images (ranging from 68 to 99 images).
• Estimated number of subjects:
  15

Study design details

• Study design:
  Prospective, observational, cross-sectional validation study conducted remotely. A pre-post design was used where 15 HCPs diagnosed 100 validated images first without assistance, then with AI support (Legit.Health Plus) to assess diagnostic accuracy improvements.
• Main study objective:
  To validate that the information provided by the Legit.Health Plus medical device increases the true diagnostic accuracy of healthcare professionals (HCPs) in the diagnosis of Generalised Pustular Psoriasis (GPP). Additionally, as a secondary objective, to validate that the device increases diagnostic accuracy for other dermatological conditions, specifically Hidradenitis Suppurativa (HS).
• Setting:
  Persons: 15 Healthcare Professionals participated (11 Primary Care Physicians and 4 Dermatologists). Inclusion criteria required board-certified physicians regardless of experience level. Place: The study was conducted in a remote setting via a dedicated web-based platform. Time Period: The study was conducted from June 01, 2024, to September 15, 2024. Selection: Participants were recruited to evaluate a dataset of 100 validated images derived from dermatology atlases and prior research, containing GPP, HS, and confounders (e.g., eczema, acne). Arms: The study utilised a single-arm, self-controlled design. Participants acted as their own control group, diagnosing images first without the device (standard clinical practice) and subsequently using the Legit.Health Plus device to confirm or revise the diagnosis.
• Comparators:
  The study did not involve an external control group. A within-subject design was employed where the physicians served as their own control. The comparator was the healthcare professional's initial unassisted diagnosis (standard clinical practice) versus their subsequent diagnosis made with the support of the Legit.Health Plus AI analysis (top 5 diagnoses with confidence levels) for the same set of images.
• Outcomes:
  Primary Outcome: Diagnostic accuracy for Generalised Pustular Psoriasis (GPP), measured as the percentage of correct diagnoses with and without the device. Secondary Outcomes: Diagnostic accuracy for Hidradenitis Suppurativa (HS) and other dermatological conditions (e.g., Palmoplantar Pustulosis, Impetigo, Acne), measured as the percentage of correct diagnoses. Impact Assessment: Categorisation of the device's influence on the physician's decision as reinforcing (maintaining correct diagnosis), improving (changing from incorrect to correct), no impact, or negative impact (changing from correct to incorrect).
• Data analysis plan:
  Analysis was conducted using Python (numpy, pandas). Diagnostic accuracy was calculated as the percentage of correct diagnoses for the total cohort and stratified by speciality (Primary Care vs. Dermatologist) and pathology. To estimate the correlation between diagnoses with and without the device, concordance was analysed. A McNemar test was performed to evaluate the statistical significance of the difference in accuracy between the unassisted and AI-assisted methods. Impact was categorised into: reinforcement, improvement, no impact, or negative impact. A p-value < 0.05 was considered statistically significant.
• Summary results:
  Overall diagnostic accuracy significantly increased from 47.91% to 62.81% (p < 0.0001) with the use of the device. This improvement was most pronounced in primary care physicians, whose accuracy rose from 44.29% to 61.71% (+17.42%), compared to dermatologists (57.25% to 65.65%).; ; For the primary target, Generalised Pustular Psoriasis (GPP), overall accuracy doubled from 23.70% to 46.67% (p=0.00001). In primary care specifically, GPP detection rates increased by 120%. For secondary targets, diagnostic accuracy for Hidradenitis Suppurativa improved from 85.48% to 93.60% (p=0.00195). Palmoplantar Pustulosis diagnosis in primary care improved by 146%. The device improved the doctor's diagnosis in 16.61% of all cases and reinforced correct diagnoses in 46.13%. No adverse events were reported.