Objective: The aims of this study were: (i) to determine the factors associated with HRQOL at baseline in our cohort, and (ii) to evaluate if there are differences in baseline HRQOL measures by antiretroviral treatment. Methods: The Short Form 36 (SF-36) was administered between 2006 and 2010 among members of the United States HIV Natural History Study cohort (NHS), and participants who completed the SF-36 were included in the study. Physical component summary (PCS) and mental component summary (MCS) scores were computed based on standard algorithms. Multivariate linear regression models were constructed for PCS and MCS to estimate the association between selected variables and HRQOL scores. Results: Antiretroviral therapy (ART) was not independently associated with HRQOL scores. Factors associated with PCS were CD4+ count < 200 cells/mm3 (β = -5.84, 95% CI: -7.63, -4.06), mental comorbidity (β = -2.82, 95% CI: -3.79, -1.85), medical comorbidity (β = -2.51, 95% CI: -3.75, -1.27), AIDS diagnosis (β = -2.38, 95% CI: -3.79, -0.98). Others were gender, military rank, marital status, and age. Factors independently associated with MCS were CD4+ count < 200 cells/mm3 (β = -1.93, 95% CI: -3.85, -0.02), mental comorbidity (β = -6.25, 95% CI: -7.25, -5.25), age (β = 0.37, 95% CI: 0.14, 0.60), and being African American (β = 1.55, 95% CI: 0.63, 2.47). Conclusion: Among military active duty and beneficiaries with HIV, modifiable factors associated with HRQOL measures included advanced HIV disease, and mental or medical comorbidity. Addressing these factors may improve quality of life of HIV-infected individuals in the NHS cohort.