Background and Objective: Point-of-care ultrasound (POCUS) has become an integral part of the curricula of various spe-cialty residency programs. POCUS education requires learners to be proficient in image acqui-sition, image interpretation, and integration into clinical practice. The assessment of POCUS skills and competency is variable. This study aimed to assess the validity and reliability of us-ing still ultrasound images as examination test items without providing clinical vignettes or anatomical locations. Methods: This is a proof-of-concept survey pilot study, residents from the Middle East who had com-pleted their training in emergency medicine were surveyed during their competency assess-ment examination. They were provided with a single ultrasound image of an abdominal aortic aneurysm in both print and PowerPoint slide format. The task was to identify the diagnosis based on this image, which was taken from a reference emergency medicine textbook. Results: The study showed that only 50% of the residents correctly identified the diagnoses based on the still images. Many of the incorrect answers included diagnoses that were not related to the anatomical structure that was scanned, indicating that the residents were not able to identify the anatomical locations, which highlighted the challenges of interpreting a single still image without clinical context and emphasized the need for valid and reliable test items in point-of-care ultrasound (POCUS) education and assessment. Conclusions: National and international emergency medicine societies should establish standards for writ-ing POCUS examination test items to ensure their validity and reliability.

Keywords: point-of-care ultrasound (POCUS); examination test item validity; examination test item reliability; objective structured clinical examination (OSCE); examination test item.