You raise a good point for the diffusion case, which trains only on positive examples, but generally speaking negative examples will warp the manifold appropriately.