Background: People with HIV with access to treatment are growing older and living healthier lives than in the past, and while health improvements and increased survival rates are welcome, the psychological and social consequences and quality of life of ageing are complex for this group. Understanding how ageing, HIV and quality of life intersect is key to developing effective interventions to improve QoL. Methods: One hundred people with HIV over the age of 50 (range 50–87, mean 58), were recruited through HIV community organizations, and clinics, and included men who have sex with men (MSM), and Black African and White heterosexual men and women. The WHOQOL-HIV BREF was used, as well as the Every Day Memory Questionnaire, and additional questions on anxiety and depression to supplement the WHOQOL. Results: While most rated their quality of life (QoL) positively, bivariate analysis showed that better QoL (total score and most domains) was strongly associated with being a man; in a relationship; in paid employment; having higher level of income; not on benefits, and to a lesser degree with being MSM, having higher level of education, and diagnosed after the age of 40. Multivariate analysis showed that not being on benefits was the variable most consistently associated with better quality of life, as was being partnered. Concerns about everyday memory difficulties, and anxiety and depression scores were strong predictors of poorer quality of life. Conclusion: While the cross-sectional nature of the investigation could not establish that the associations were causal, the findings indicate that concerns about memory difficulties, anxiety and depression, as well as gender, ethnicity, financial factors, and relationship status, are important contributors to QoL in this group. These findings point towards the need for further research to clarify the mechanisms through which the factors identified here affect QoL, and to identify possible interventions to improve the QoL of older people living with HIV.