prehistoric-systems/omnivore
1
0
Fork 0
mirror of https://github.com/omnivore-app/omnivore.git synced 2026-03-11 08:54:26 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
4820 commits 201 branches 31 tags 432 MiB
Commit graph

71 commits

Author SHA1 Message Date
Hongbo Wu
3a3a06b5d6 Fix vote count 2022-11-29 16:18:09 +08:00
Hongbo Wu
1fe86b6230 Parse owner 2022-11-29 15:48:45 +08:00
Hongbo Wu
f8e0efad05 fix bug 2022-11-29 15:40:22 +08:00
Hongbo Wu
8ee19a98fa parse authors 2022-11-29 12:26:37 +08:00
Hongbo Wu
c993fa11be Add stack-overflow content-handler 2022-11-29 11:39:23 +08:00
Jackson Harper
db4f225b5c
Merge pull request #1426 from omnivore-app/fix/github-handler-titles 
Use greedy matching on the GitHub title regex
 2022-11-17 13:42:23 +08:00
Hongbo Wu
c18217e697 Fix logRecord being shared by requests 2022-11-17 11:28:46 +08:00
Hongbo Wu
d9237723c5 Share puppeteer browser instance between content handlers 2022-11-17 11:28:46 +08:00
Hongbo Wu
e866541ae1 Fix puppeteer launch in head mode 2022-11-17 11:28:46 +08:00
Hongbo Wu
8bdbd86e97 Make headless = false in local testing 2022-11-17 11:27:51 +08:00
Jackson Harper
0d83ecb068 Use greedy matching on the GitHub title regex 
This fixes issues with titles that have slashes (/) in their title
as the regex match would look for the last possible matching
slash.
 2022-11-16 13:52:58 +08:00
Hongbo Wu
c8febe900f Fix prettier error 2022-11-16 10:34:52 +08:00
Jackson Harper
726f69484f Better content extraction of title and description for GitHub URLs 2022-11-15 18:43:55 +08:00
Hongbo Wu
9d753e3c0e Forward convertkit confirmation email 2022-11-02 15:27:30 +08:00
Hongbo Wu
ae3ff6f34b Add test case 2022-11-01 17:48:39 +08:00
Hongbo Wu
67082a478e Add support for hey world newsletters 2022-11-01 16:41:42 +08:00
Jackson Harper
8712a3efef
Merge pull request #1373 from omnivore-app/fix/substack-embedded-tweets 
Handle embedded tweets in substack emails
 2022-11-01 15:04:03 +08:00
Jackson Harper
e1d7eeed01 Make test verify preparse is true 2022-11-01 10:46:31 +08:00
Jackson Harper
9b29a3c261 querySelectorAll returns a NodeList which we can iterate 2022-11-01 10:34:18 +08:00
Jackson Harper
392ac15a62 Treat all tweets as elements so we dont need to cast 2022-11-01 10:31:12 +08:00
Jackson Harper
a4f0698974 Handle older style substack image embeds as well as new 2022-10-31 21:57:29 +08:00
Jackson Harper
734c4337df Add missing test file 2022-10-31 21:51:22 +08:00
Jackson Harper
2622437831 Fix linting 2022-10-31 21:42:31 +08:00
Jackson Harper
cc91e43572 Handle embedded tweets in substack emails 
This does a few things:
- tags static tweets found in substack emails with a special class
- upgrades readability to ignore special class names
- reduces some readability debug output
 2022-10-31 21:28:36 +08:00
Hongbo Wu
a7b208fd26 Skip non-author replies in twitter thread 2022-10-31 16:51:20 +08:00
Hongbo Wu
1067747cc2 Add twitter author when saved 2022-10-27 19:02:16 +08:00
Hongbo Wu
a536ed9b5e Save newer thread with puppeteer too 2022-10-27 14:20:20 +08:00
Hongbo Wu
b556bc17e8 Set navigation timeout to be 60 seconds 2022-10-27 13:05:35 +08:00
Hongbo Wu
6ca1e7dc25 Add no-sandbox in the puppeteer args 2022-10-27 12:49:00 +08:00
Hongbo Wu
03777587f2 Fetch max 100 tweets in a thread 2022-10-26 22:44:19 +08:00
Hongbo Wu
b5926ccf1c Get old tweet thread with puppeteer and new tweet with twitter api 2022-10-26 20:41:51 +08:00
Hongbo Wu
5f0d0ed69b Get tweet reply ids from thread id using puppeteer 2022-10-25 22:09:16 +08:00
Hongbo Wu
343c8fb5cc Get tweet reply ids from thread id 2022-10-25 18:25:49 +08:00
Hongbo Wu
31bc9fe5be Fix node-weekly newsletter getting forwarded 
* This error is caused by a previous rebase conflict and resolving the conflict caused an important line missing
* Revert the change to add CooperPressHandler to the array of newsletter handlers
* Updated the test to get the correct handler from email content to prevent such cases
 2022-10-21 09:57:05 +08:00
Hongbo Wu
57846a1c5e Fix tests 2022-10-18 15:14:38 +08:00
Hongbo Wu
706607ea2b Rebase main 2022-10-18 14:54:33 +08:00
Hongbo Wu
1fda6946bb Save newsletters hosted by cooper-press.com correctly 2022-10-18 14:53:02 +08:00
Hongbo Wu
9757394a05
Merge pull request #1318 from omnivore-app/fix/newsletter-ghost-org 
Save newsletters hosted by ghost.org correctly
 2022-10-18 14:50:27 +08:00
Jackson Harper
59427fa896 Mock the substack domain also when testing redirects 
We were making the actual call to the redirected URL here,
instead of mocking it. We need to mock both the embedded email
domain, and the domain we are redirected to.
 2022-10-18 12:51:53 +08:00
Hongbo Wu
d8c5c97960 Save newsletters hosted by ghost.org correctly 2022-10-17 19:27:02 +08:00
Hongbo Wu
37439c6622 Save fs.blog newsletter correctly 2022-10-17 17:59:27 +08:00
Jackson Harper
ff21770a70 Add GitHubHandler class 2022-10-13 18:38:01 +08:00
Jackson Harper
f39c6923b0 Create a content handler for GitHub pages 
This will search for the markdown <article> element and make it
the only page element so we add a repo's README.
 2022-10-13 18:11:14 +08:00
Hongbo Wu
5dcde8dac3 Fix not correctly identify milkroad newsletters due to the HTML format changes again by the author 2022-10-13 10:52:03 +08:00
Hongbo Wu
08c30f9c24 Show the original tweet firstly 2022-10-10 15:33:36 +08:00
Hongbo Wu
5d70d04154 If the saved tweet is a reply, we need to get the referenced tweet 2022-10-10 15:12:41 +08:00
Hongbo Wu
220576235f
Merge pull request #1291 from omnivore-app/sywhb-patch-1 
Get tweet thread by conversation_id
 2022-10-10 09:52:30 +08:00
Hongbo Wu
8f048dd15e Change the milkroad newsletters parser according to the change of milkroad newsletter 2022-10-08 09:37:12 +08:00
Hongbo Wu
06c1e09505 lint 2022-10-08 09:28:50 +08:00
Hongbo Wu
2ff995d3b2
Get tweet thread by conversation_id 2022-10-08 09:09:18 +08:00