Dyson spheres are hypothetical megastructures built by advanced extraterrestrial civilizations to harvest radiation energy from stars. Here, we combine optical data from Gaia DR2 with mid-infrared data from AllWISE to set the strongest upper limits on the prevalence of partial Dyson spheres within the Milky Way. Conservative upper limits are presented on the fraction of stars that may potentially host non-reflective Dyson spheres that absorb up to 90% of the bolometric luminosity of their host stars and emit thermal waste-heat in the 100–1000 K range. In our sample of stars within 100 pc, we find that one over 100000 stars could potentially host ∼ 300 K Dyson spheres at 90% completion. These limits become progressively weaker for less complete Dyson spheres due to increased confusion with naturally occurring sources of strong mid-infrared radiation, and also at larger distances, due to the detection limits of WISE.