No, that is how natural geothermal energy works. Perhaps you mistakenly thought I was saying the heat comes from sunlight? I didn't. The heat comes from below (or, in some cases, from internal radioactive decay). And this delivery of heat from below (or from decay) is a slow process, taking a very long time, which is why geothermal resources have to be buried deeply (otherwise, that heat just leaks out and the temperature of the geothermal resource is too low).
Yeah, "accumulate the heat over thousands of years" indeed sounds a bit misleading to me. The heat is largely already there (or is generated pretty uniformly through radioactive processes), it's just slowly transmitted outwards down a gradient.
No, the heat is not already there. The heat comes in and goes out; the heat energy initially in the crust decays away exponentially with time and has no effect on the steady stage temperature gradient.
It was not initially in the rocks that we are tapping for geothermal energy, which would be a few kilometers. I wasn't talking about the Earth as a whole. Remember, this is about why so much more thickness is needed for the rocks for ordinary geothermal energy systems, vs. artificial geothermal.