Features extracted from cardiac MRI (CMR) are correlated with cardiovascular disease outcomes such as aneurysm, and have a substantial heritable component. To determine whether disease-relevant measurements are feasible in non-cardiac specific MRI, and to explore their associations with disease outcomes, and genetic and environmental risk factors. We segmented the heart, aorta, and vena cava from abdominal MRI scans using deep learning, and generated six image-derived phenotypes (IDP): heart volume, four aortic and one vena cava cross-sectional areas (CSA), from 44,541 UK Biobank participants. We performed genome- and phenome-wide association studies, and constructed a polygenic risk score for each phenotype. We demonstrated concordance between our IDPs and related IDPs from CMR, the current gold standard. We replicated previous findings related to sex differences and age-related changes in heart and vessel dimensions. We identified a significant association between infrarenal descending aorta CSA and incident abdominal aortic aneurysm, and between heart volume and several cardiovascular disorders. In a GWAS, we identified 72 associations at 59 loci (15 novel). We derived a polygenic risk score for each trait and demonstrated an association with TAA diagnosis, pointing to a potential screening method for individuals at high-risk of this condition. We demonstrated substantial genetic correlation with cardiovascular traits including aneurysms, varicose veins, dysrhythmia, and cardiac failure. Finally, heritability enrichment analysis implicated vascular tissue in the heritability of these traits. Our work highlights the value of non-specific MRI for exploring cardiovascular disease risk in cohort studies.