7 Commits

Author	SHA1	Message	Date
Arik Jones (aider)	01465a08b7	fix: set default values for requests_per_second and burst_limit in configuration to prevent rate limiter errors	2024-09-30 14:19:00 -05:00
Arik Jones (aider)	ee1561c502	feat: add LinksContainerSelector to SiteConfig and enhance scraping logic with depth control and link extraction	2024-09-30 14:05:10 -05:00
Arik Jones	333b9a366c	fix: Resolve playwright function deprecations and io/ioutil function deprecations.	2024-09-24 15:13:36 -05:00
Arik Jones	73116e8d82	Fix logging and other issues from preventing scraping	2024-09-21 15:54:33 -05:00
Arik Jones (aider)	53dcd6eb71	feat: Add support for exclusionary CSS paths in config.go	2024-09-14 20:59:08 -05:00
Arik Jones (aider)	52c7de255d	feat: Implement scraping of multiple URLs with optional CSS locators and separate output files	2024-09-14 20:35:35 -05:00
Arik Jones	0163c4e504	Adds a configuration layer for use rollup.yml which may be preferred over CLI flags.	2024-09-05 23:41:39 -05:00