I'm a big fan of modern JavaScript frameworks, but I don't fancy SSR, so have be... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		unlog on June 10, 2024 \| parent \| context \| favorite \| on: Show HN: Crawl a modern website to a zip, serve th... I'm a big fan of modern JavaScript frameworks, but I don't fancy SSR, so have been experimenting with crawling myself for uploading to hosts without having to do SSR. This is the result

mikeqq2024 on June 11, 2024 [–]

for a long crawling task, if exited/broken for any reason, does it save and restore at the next run?

eXpl0it3r on June 11, 2024 | [–]

The README says:

> Can resume if process exit, save checkpoint every 250 urls

mikeqq2024 on June 11, 2024 | | [–]

nice, better make it as a commandline option with default value. 250 is too many for large files and slow connection.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact