Very good point. I read Olaf Stapledon's First and Last man, a sci-fi book which describes the next few billion years for mankind and deals with this issue.
In one scenario, human society collapses, spends 80 million years living as hunter/gatherers (with some evolution) and then develops once again to a technological civilization, in part because the minerals and fossil fuels have been replenished.
In one scenario, human society collapses, spends 80 million years living as hunter/gatherers (with some evolution) and then develops once again to a technological civilization, in part because the minerals and fossil fuels have been replenished.