This paper addresses the problem of reconstructing an image from low-count Positron Emission Tomography (PET) data. We build on recent advances combining deep neural networks with expectation-maximization algorithms. More specifically, we extend the recent DIPRecon approach [1] to handle various challenges linked to natively low-count Yttrium 90 data. To this end, we rely on the interpretation of the Deep Image Prior (DIP) in the light of approximate Bayesian inference. By introducing a stochastic gradient Langevin dynamics (SGLD) optimizer, we reduce the tendency of the algorithm to overfit the noisy maximum likelihood estimate while improving the contrast recovery figures. Moreover, as a by-product of the SGLD optimization, the method recovers an uncertainty value associated with every voxel in the estimated image. We qualitatively and quantitatively evaluate the proposed method on data acquired with the NEMA IEC body phantom achieving high-quality results.