I have bought a lot of books from Amazon, and if you look past the scrubbing with a marker, it's always some library within America - so yes even books from Amazon are just books taken from public libraries.
Like the other poster said, that's what libraries do when they're done with books. I help at my kid's school library, mostly shelving books, but also processing discards, which is remove from catalog, remove or mark over the barcode and library stamps so the book doesn't come back, then the librarian has to do some procedural junk and if nobody at the school wants the books, they get sold to a used book wholesaler and then hopefully to someone who wants it. The wholesalers don't pay much, but it helps buy new books the discarding has made room for.
That's not necessarily a smoking gun; public libraries often give away old books they don't want to keep around anymore. About a quarter of the books in my personal library are ex-library books, but none were stolen.
Of course if it's some popular in-demand book, it was almost certainly stolen. Most of the free library books I've scored are 50 year old textbooks about obscure/obsolete topics.