prehistoric-systems/omnivore
1
0
Fork 0
mirror of https://github.com/omnivore-app/omnivore.git synced 2026-03-11 08:54:26 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
2441 commits 201 branches 31 tags 432 MiB
Commit graph

25 commits

Author SHA1 Message Date
Hongbo Wu
37075f076e Remove userDataDir 2022-06-29 22:56:14 +08:00
Jackson Harper
feb197c731 Fix a crash when parsing content fetches that are blocked 2022-06-22 14:55:37 -07:00
Hongbo Wu
486f22a594 remove redundant async 2022-06-15 22:31:55 +08:00
Hongbo Wu
2aafd39650 add fastcompany.com to the non-script hosts list 2022-06-15 22:27:25 +08:00
Hongbo Wu
e028e2e440 generate test page for fast company 2022-06-15 21:22:25 +08:00
Jackson Harper
486f3c930b Remove PROXY_URL from content-fetch 2022-06-14 20:30:02 -07:00
Jackson Harper
be2801477b Add some extra debugging 2022-06-14 16:13:43 -07:00
Jackson Harper
159a7f8950 Fallback to scrapingbee if a page cant fetch content 2022-06-14 16:06:01 -07:00
Jackson Harper
814f6098a3 Log proxy url 2022-06-14 14:27:19 -07:00
Jackson Harper
a4ad78652a Allow specifying a proxy url when launching puppeteer 2022-06-14 13:30:32 -07:00
Jackson Harper
b94215f1fc Allow selectively disabling javascript on some hosts 
Some hosts readability is improved by disabling javascript
 2022-06-10 13:25:14 -07:00
Jackson Harper
cb98a9cf86 Make clients opt into creating a page when uploading a file 2022-05-26 21:40:40 -07:00
Hongbo Wu
0cc7e84a82 Fix content not getting parsed by linkedom properly without <html> tag by replacing innerHtml with outerHtml 2022-05-18 15:52:16 +08:00
Hongbo Wu
8f0447ed3f Stop blocking images and css file 2022-05-18 15:50:52 +08:00
Hongbo Wu
745f55a843 Set headless=true 2022-05-14 10:47:15 +08:00
Hongbo Wu
80c14cd6ca Remove single-process from chromium args 2022-05-14 10:37:06 +08:00
Hongbo Wu
f5003c1370 Stop blocking script 2022-05-13 12:17:19 +08:00
Hongbo Wu
37e55add98 Stop blocking stylesheet and media 2022-05-13 12:09:05 +08:00
Hongbo Wu
60bbbb6cf3 Block requests to 'font', 'image', 'stylesheet', 'script', 'media' in puppeteer 2022-05-12 17:10:38 +08:00
Hongbo Wu
9606cd6b28 Remove chrome-aws-lambda dependencies 2022-05-12 16:32:22 +08:00
Hongbo Wu
0984dca183 Remove adblocker and block resources by url and also block mathJax script 2022-05-11 22:04:47 +08:00
Hongbo Wu
45a2a1393f Use chrome-aws-lambda args 2022-05-09 20:59:20 +08:00
Hongbo Wu
47c3578820 Add more puppeteer args 2022-05-09 14:32:46 +08:00
Hongbo Wu
6f82499deb Merge puppeteer changes 2022-05-09 13:52:40 +08:00
Jackson Harper
fd934a949e Add a standalone service content fetching 2022-05-08 21:52:36 -07:00