Self-efficacy is a personal belief in one's capability to successfully execute the behaviors necessary to attain designated types of performances. Sometimes described as task-specific self-confidence, self-efficacy is a key component in many contemporary theories of motivation and learning. The purpose of this study was to develop a survey for measuring students' medical skills self-efficacy and to collect reliability and validity evidence for the instrument. A secondary purpose was to explore differences in students' self-efficacy from year 1 of medical school to year 4. We created 19 survey items based on the 6 core competencies of the Accreditation Council for Graduate Medical Education, and we collected data from 304 medical students. Results from an exploratory factor analysis suggested three interpretable factors: patient care self-efficacy (eight items, Cronbach's α = 0.92), interpersonal skills self-efficacy (three items, Cronbach's α = 0.76), and evidence-based medicine self-efficacy (three items, Cronbach's α = 0.79). We then compared students' self-efficacy at different stages of training using a one-way multivariate analysis of variance. Consistent with our expectations, we found several statistically significant differences, suggesting students' self-efficacy increased considerably from year 1 of medical school to year 4, F(9, 725) = 30.58, p < 0.001, Wilks' γ = 0.46. Using this survey, medical educators and researchers have a psychometrically sound tool for measuring students' medical skills self-efficacy during undergraduate medical education. Practical implications and future directions are discussed.