prehistoric-systems/omnivore
1
0
Fork 0
mirror of https://github.com/omnivore-app/omnivore.git synced 2026-03-11 08:54:26 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
12057 commits 201 branches 31 tags 432 MiB
Commit graph

2536 commits

Author SHA1 Message Date
Hongbo Wu
9769eab5dc fix youtube transcript 2024-05-16 11:23:36 +08:00
Hongbo Wu
8cfa24a847 allow downloading/uploading readable content 2024-05-15 21:46:22 +08:00
Hongbo Wu
1a973faadc make author optional 2024-05-15 18:12:54 +08:00
Hongbo Wu
396ada28b8 pre-cache thumbnail 2024-05-15 17:49:31 +08:00
Hongbo Wu
9f1babf787 select used columns 2024-05-15 17:42:53 +08:00
Hongbo Wu
505e3666b6 fix: add author to the chapter in digest 2024-05-15 17:38:24 +08:00
Hongbo Wu
6d0edbd61a
Merge pull request #3839 from omnivore-app/feature/upload-original-content 
feature/upload original content
 2024-05-15 16:47:46 +08:00
Hongbo Wu
950b42899d fix mock cloud storage 2024-05-15 15:53:42 +08:00
Jackson Harper
95796607d0
Merge pull request #3956 from omnivore-app/fix/update-openai 
Upgrade to gpt-4o
 2024-05-15 12:57:46 +08:00
Jackson Harper
74bf05cf06 Linting 2024-05-15 12:41:12 +08:00
Jackson Harper
f217c6f4ad Cast digestConfig to any to serialize properly 2024-05-15 12:08:29 +08:00
Jackson Harper
c6bf64a648 Upgrade to gpt-4o 2024-05-15 11:44:23 +08:00
Jackson Harper
89fb6255a5
Merge pull request #3952 from omnivore-app/fix/ios-digest-new-users 
Allow opt into digest on the library view
 2024-05-15 11:10:38 +08:00
Hongbo Wu
cca7a21884 fix tests 2024-05-15 11:02:18 +08:00
Jackson Harper
2cb1b8ed1e Add an extra test for clearing the user personalization data 2024-05-15 10:35:58 +08:00
Hongbo Wu
69608119a8 allow preflight api calls to the digest POST api 2024-05-15 09:48:42 +08:00
Jackson Harper
bbf3de1f86 Transform digest config to a JSON string before setting 2024-05-15 00:06:04 +08:00
Hongbo Wu
b7886b8d25 fix tests 2024-05-14 22:47:10 +08:00
Hongbo Wu
9dee510be1 fix rss 2024-05-14 20:18:18 +08:00
Jackson Harper
3c24dc2e78 GQL requires this be an input object 2024-05-14 18:01:18 +08:00
Hongbo Wu
dbd7b7932f cont 2024-05-14 17:23:56 +08:00
Hongbo Wu
d796503fec resolve rebase conflicts 2024-05-14 17:21:50 +08:00
Hongbo Wu
cce5f2463d still use redis for cache 2024-05-14 17:16:26 +08:00
Hongbo Wu
80968472d8 fix bug 2024-05-14 17:15:52 +08:00
Hongbo Wu
eddf9206d0 do not store original content in db 2024-05-14 17:15:51 +08:00
Hongbo Wu
6cfb06c226 normalize file url 2024-05-14 17:14:41 +08:00
Hongbo Wu
5bd157ca25 hash url as the key 2024-05-14 17:14:41 +08:00
Hongbo Wu
7a0b2f3d33 upload file only not exists 2024-05-14 17:14:41 +08:00
Hongbo Wu
9286174ec7 upload and download original content from GCS 2024-05-14 17:14:40 +08:00
Hongbo Wu
bc1c48da4b fix failed to get thumbnail from rss feed 2024-05-14 17:14:02 +08:00
Jackson Harper
c3c6083239 Use DigestConfig type when setting user personalization 2024-05-14 17:03:09 +08:00
Jackson Harper
2b9851cbdb Make digestconfig a proper type so Swift can interact with it 2024-05-14 16:49:16 +08:00
Hongbo Wu
cbba54111f
Merge pull request #3947 from omnivore-app/fix/content-api 
fix/content api
 2024-05-13 19:14:35 +08:00
Hongbo Wu
302efc119e fix tests 2024-05-13 19:13:55 +08:00
Hongbo Wu
0a96057bda fix: text not highlighted in highlightedMarkdown content 2024-05-13 18:35:24 +08:00
Jackson Harper
3514595d83
Merge pull request #3937 from omnivore-app/fix/digest-eligibility 
Fix logic on the digest eligibility test
 2024-05-13 16:29:00 +08:00
Jackson Harper
5dbed8c6c2
Merge pull request #3943 from omnivore-app/feat/api-digest-label 
Add a label to saved digests
 2024-05-13 16:28:45 +08:00
Hongbo Wu
e37abb47a1
Merge pull request #3942 from omnivore-app/fix/puppeteer-timeout 
fix: puppeteer protocol timeout
 2024-05-13 16:01:44 +08:00
Hongbo Wu
64e4258d89
Merge pull request #3935 from omnivore-app/feature/get-content-api 
feature/get content api
 2024-05-13 15:27:34 +08:00
Jackson Harper
92e3f23ec6 Add a label to saved digests 2024-05-13 15:16:30 +08:00
Hongbo Wu
34c214e7d8 fix docker error 2024-05-13 15:11:37 +08:00
Hongbo Wu
6bb81dd5c3 skip uploading if file already exists 2024-05-11 11:30:41 +08:00
Jackson Harper
7e5b0864b3 Fix logic on the digest eligibility test 2024-05-10 22:50:22 +08:00
Hongbo Wu
6f7c58289a fix cors 2024-05-10 17:48:47 +08:00
Hongbo Wu
8034e18825 fix tests 2024-05-10 16:40:55 +08:00
Hongbo Wu
6e7a436cea POST content API 2024-05-10 16:26:42 +08:00
Hongbo Wu
0f184c4c21 add get content api 2024-05-10 15:43:15 +08:00
Jackson Harper
e96d07fb40 Clean up digest email formatting 2024-05-10 15:09:45 +08:00
Jackson Harper
e749c8dc3a Add authtrx when checking user digest eligibility 2024-05-10 14:52:23 +08:00
Jackson Harper
500ec889c8 Add block around var definition 2024-05-10 14:48:27 +08:00
Hongbo Wu
01ebcbb16b add bulk upload original content job 2024-05-10 14:37:05 +08:00
Jackson Harper
e85bb9aa6a Only allow eligible users to enable digest 2024-05-10 14:34:52 +08:00
Hongbo Wu
4c48a44512 format summary of digest in library and email 2024-05-09 17:38:21 +08:00
Hongbo Wu
ab85805e24
Merge pull request #3926 from omnivore-app/feature/delete-task-api 
feature/delete task api
 2024-05-09 15:15:33 +08:00
Hongbo Wu
bd92d0847b add some logs 2024-05-09 15:15:22 +08:00
Hongbo Wu
922adbae42 add delete digest api 2024-05-09 14:02:10 +08:00
Hongbo Wu
431dc420a9 add delete task api 2024-05-09 12:43:37 +08:00
Hongbo Wu
17708915d9 wrap content in <html><body> element 2024-05-09 11:46:01 +08:00
Hongbo Wu
e1990258f1 send analytic events for each digest channel 2024-05-09 11:38:45 +08:00
Hongbo Wu
5a730f12e7 only include the "Read in Omnivore" button if the digest is saved in library 2024-05-09 10:53:57 +08:00
Hongbo Wu
90b0845909 add "Update digest preference" button to the digest email 2024-05-09 10:45:13 +08:00
Jackson Harper
279874f319
Merge pull request #3917 from omnivore-app/fix/usernames-add-posts 
Add posts and digest(s) to reserved names
 2024-05-08 21:07:37 +08:00
Hongbo Wu
edf21c8078 add digest to the reserved username list 2024-05-08 17:08:53 +08:00
Hongbo Wu
42b8aede0e randomly pick the qualified thumbnail 2024-05-08 17:04:19 +08:00
Hongbo Wu
ee2ca1ec97 add a "read in omnivore" button to the digest email 2024-05-08 16:49:12 +08:00
Jackson Harper
4169aa0f71 Add posts and digest(s) to reserved names 2024-05-08 16:14:57 +08:00
Hongbo Wu
5f4ed6ad11 add move digest to library api 2024-05-08 15:58:29 +08:00
Hongbo Wu
5d04537bd7 fix tests 2024-05-08 14:36:34 +08:00
Hongbo Wu
9259913344 allow saving digest in the library as a channel 2024-05-08 14:32:16 +08:00
Hongbo Wu
d880a6c37d send json response 2024-05-08 14:06:40 +08:00
Hongbo Wu
5c91e66b47 catch db error in digest job and only send notifications when digest created 2024-05-07 19:17:40 +08:00
Hongbo Wu
31233d8348 get client from user-agent 2024-05-07 14:53:29 +08:00
Hongbo Wu
23eae7871a send client info to the analytics 2024-05-07 12:30:51 +08:00
Hongbo Wu
bc802adc93 feat: send digestId with the push notification for deep link 2024-05-06 16:44:21 +08:00
Hongbo Wu
2fd3b930fe use the shortened title as subject 2024-05-03 16:06:50 +08:00
Hongbo Wu
e013963143 update email content 2024-05-03 15:56:19 +08:00
Hongbo Wu
6a0ea1b078 update gcs upload filename 2024-05-03 15:15:11 +08:00
Hongbo Wu
9bac4798b1 put omnivore url in each chapter 2024-05-03 14:56:45 +08:00
Hongbo Wu
7b2e336eca log unknown channel 2024-05-03 14:45:01 +08:00
Hongbo Wu
a9dd2d0d4c dedupe channels 2024-05-03 14:43:18 +08:00
Hongbo Wu
0692ab0454 add chapters to the digest email 2024-05-03 14:39:12 +08:00
Hongbo Wu
20408277d2 add digestConfig to the userPersonalization API request data 2024-05-03 14:35:55 +08:00
Hongbo Wu
7060ae771c send notifications to the channels defined in digest config 2024-05-03 14:17:22 +08:00
Hongbo Wu
3e84c2f1df fix tests 2024-05-03 12:17:07 +08:00
Hongbo Wu
0639a52676 add digestConfig to the userPersonalization entity 2024-05-03 11:13:24 +08:00
Hongbo Wu
6e554e2bca update upload file name 2024-05-02 19:10:02 +08:00
Hongbo Wu
4610c81092 upload summaries to gcs 2024-05-02 19:00:21 +08:00
Hongbo Wu
298a50fba2 allow random model 2024-05-02 18:33:41 +08:00
Hongbo Wu
49bdf0a6f4 select model from prompt and defaults to openai 2024-05-02 18:30:04 +08:00
Jackson Harper
9a9147652f We dont need the newsletter classification email anymore 2024-05-02 13:40:45 +08:00
Hongbo Wu
d589c18b68 generate a unique id for each scheduled digest to avoid duplication 2024-04-30 12:48:19 +08:00
Hongbo Wu
8edfce596b revert to batch api call 2024-04-30 12:48:19 +08:00
Hongbo Wu
83ed971b28 improve logs 2024-04-30 12:48:19 +08:00
Hongbo Wu
aa30b592b5 reduce debug logs 2024-04-30 12:48:19 +08:00
Hongbo Wu
debb396b44 better error handling 2024-04-30 12:48:19 +08:00
Hongbo Wu
ea227b6462 use claude 2024-04-30 12:48:19 +08:00
Hongbo Wu
0a2f9dba25 filter out summary if longer than 1000 words 2024-04-30 12:48:19 +08:00
Hongbo Wu
cbf1f97630 fix timer on get candidates function 2024-04-30 12:48:19 +08:00
Hongbo Wu
9a6ec7a58b filter out summary if shorter than 200 2024-04-30 12:48:19 +08:00
Hongbo Wu
3954fa309b randomly select at most 25 candidates and time each step 2024-04-30 12:48:19 +08:00
Hongbo Wu
32be8126df reduce number of db calls for search library items 2024-04-30 12:48:19 +08:00
Hongbo Wu
14e91d338d do not create user profile and topics 2024-04-30 12:48:19 +08:00
Hongbo Wu
d0d088aae3 fix: remove redundant user query 2024-04-30 12:48:19 +08:00
Hongbo Wu
cfc1d12835
Merge pull request #3866 from omnivore-app/feature/data-loader 
feat: use dataloader for labels and highlights
 2024-04-29 15:20:53 +08:00
Jackson Harper
4632d08aa0
Merge pull request #3863 from omnivore-app/fix/fetch_feed 
increase rss parser timeout to 20 seconds
 2024-04-28 22:45:55 -07:00
Jackson Harper
c70c36ec7d
Merge pull request #3826 from kimlimjustin/feat/increase-daily-upload-limit 
feat: increase daily upload limit
 2024-04-28 22:45:10 -07:00
Hongbo Wu
8549c16ceb fix import 2024-04-28 21:55:09 +08:00
Hongbo Wu
ef2ff36f0e fetch api key once 2024-04-28 21:51:30 +08:00
Hongbo Wu
a78d438e21 fix: findOne with relations will execute query twice 2024-04-28 15:58:55 +08:00
Hongbo Wu
7ef6973a47 load labels when getting item 2024-04-28 14:25:11 +08:00
Hongbo Wu
89d6564d74 increase log metadata size to 10000 characters 2024-04-26 15:18:34 +08:00
Hongbo Wu
342db3928c batching upload_files 2024-04-26 14:44:17 +08:00
Hongbo Wu
01246207cb fix logger 2024-04-26 12:06:20 +08:00
Hongbo Wu
66d7649620 fix tests 2024-04-26 11:54:55 +08:00
Hongbo Wu
2231f03948 reduce joins 2024-04-26 11:36:16 +08:00
Hongbo Wu
364f8f102f use enum 2024-04-26 11:24:07 +08:00
Hongbo Wu
6493b8e2f6 batching recommendations 2024-04-26 11:17:52 +08:00
Hongbo Wu
fbe816d2d2 convert content if requested 2024-04-26 11:07:41 +08:00
Hongbo Wu
c7ecbd53f8 put dataloaders in apollo context 2024-04-25 23:26:55 +08:00
Hongbo Wu
2904c321e9 batching highlights 2024-04-25 23:03:08 +08:00
Hongbo Wu
1445dee933 batching labels 2024-04-25 22:20:56 +08:00
Hongbo Wu
83526e1b56 increase rss parser timeout to 20 seconds 2024-04-25 10:55:27 +08:00
Jackson Harper
636fa17712 Default to 5 items 2024-04-24 16:39:46 -07:00
Jackson Harper
ae8790b545
Merge pull request #3848 from omnivore-app/feat/ios-digest-view 
iOS Explain and Digest views
 2024-04-24 16:13:50 -07:00
Jackson Harper
3dcc514be7
Merge pull request #3835 from omnivore-app/fix/guess-user-email 
return UNKNOWN as error code if email exists when user signs up
 2024-04-24 16:06:46 -07:00
Jackson Harper
5a6d57e3da Remove uneeded 2024-04-24 15:58:39 -07:00
Hongbo Wu
0c0a95a79c fix newsletter dir not saved correctly 2024-04-24 21:10:13 +08:00
Hongbo Wu
e788c71eff rename variables 2024-04-24 16:49:46 +08:00
Hongbo Wu
e680202b7e better error message 2024-04-24 16:17:05 +08:00
Hongbo Wu
824b256d20 fix memory leak from axios error 2024-04-24 15:55:54 +08:00
Jackson Harper
e349f6c675
Merge pull request #3853 from omnivore-app/fix/android-obsidian-cors 
Add localhost to allow android Obsidian to query the API
 2024-04-23 08:01:37 -07:00
Jackson Harper
dfdb04af92 Add localhost to allow android Obsidian to query the API 2024-04-23 07:57:41 -07:00
Hongbo Wu
3fb7193fb5 reduce getTextNodesBetween timeout to 60 seconds 2024-04-23 21:28:02 +08:00
Hongbo Wu
002e455bbd fix tests 2024-04-23 17:54:52 +08:00
Hongbo Wu
0faafa0786 fix rebase conflicts 2024-04-23 16:04:59 +08:00
Hongbo Wu
2f7c18b363 add more logs 2024-04-23 15:50:15 +08:00
Hongbo Wu
1b6f4976d3 remove timeout 2024-04-23 15:50:15 +08:00
Hongbo Wu
de258ae0bf timeout if job takes more than 10 minutes 2024-04-23 15:50:15 +08:00
Hongbo Wu
a77ade0890 fix: export all items job stuck 2024-04-23 15:50:14 +08:00
Hongbo Wu
201a5817ef
Merge pull request #3843 from omnivore-app/fix/newsletter-language 
fix language and dir not saved for newsletters
 2024-04-23 14:45:36 +08:00
Jackson Harper
2449c872bf Add Obsidian's capacitor origin to our CORs config 2024-04-22 19:15:20 -07:00
Jackson Harper
cd5940b840 Feature flag explain 2024-04-22 19:04:10 -07:00
Jackson Harper
f0be6a41a9 Clean up, remove old stubbed stuff 2024-04-22 17:40:50 -07:00
Jackson Harper
7f0a95b454 Add iOS digest views 2024-04-22 15:52:28 -07:00
Hongbo Wu
dcf7c570c4 fix tests 2024-04-22 10:57:48 +08:00
Hongbo Wu
9f65588ba0 fix language and dir not saved for newsletters 2024-04-21 10:41:29 +08:00
Hongbo Wu
8501509564 include content in updateSince api response by default for offline use 2024-04-19 14:26:17 +08:00
Hongbo Wu
fa4b3bf462 fix cron schedule bug with bull 2024-04-19 14:25:18 +08:00
Hongbo Wu
ac775ae8f2 Drain httpServer when apollo server stops 2024-04-19 14:25:18 +08:00
Hongbo Wu
273ff9d4be increase max ai-digest beta users to 10 2024-04-19 14:25:18 +08:00
Hongbo Wu
07475e69a4 change cron to schedule at 10:30 2024-04-19 14:25:18 +08:00
Hongbo Wu
a69d1240c8 add urlsToAudio back 2024-04-19 14:25:18 +08:00
Hongbo Wu
efa930a31a send push notifications when digest is ready 2024-04-19 14:25:18 +08:00
Hongbo Wu
7871154160 mark failed task 2024-04-19 14:25:18 +08:00
Hongbo Wu
b93573460c filter out summary if the length is longer than the original content 2024-04-19 14:25:18 +08:00
Hongbo Wu
099310ed95 reuse existing candidates if there is no one 2024-04-19 14:25:18 +08:00
Hongbo Wu
0d8b17f6aa remove urlsToAudio from digest 2024-04-19 14:25:18 +08:00
Hongbo Wu
1da9b51dec do not reuse candidates after refreshing 2024-04-19 14:25:18 +08:00
Hongbo Wu
09ce46e7ea include readable content in search api response by default 2024-04-19 11:30:09 +08:00
Hongbo Wu
22850bb6ab return UNKNOWN if email exists when user signs up 2024-04-18 14:04:38 +08:00
Jackson Harper
9ed94b4571
Merge pull request #3796 from omnivore-app/feature/captcha 
feature: add captcha to email-login and reset-password pages
 2024-04-18 09:48:08 +08:00
Hongbo Wu
26579d9cbb use ### for title in the digest content 2024-04-17 18:26:48 +08:00
Hongbo Wu
3e51aa7c03 generate description 2024-04-17 16:49:19 +08:00
Hongbo Wu
0eb6460bec fix tests 2024-04-17 16:35:29 +08:00
Hongbo Wu
aeda4f5d5d add libraryItemIds to the payload 2024-04-17 16:20:00 +08:00
Hongbo Wu
57cb897839 generate content 2024-04-17 15:45:05 +08:00
Hongbo Wu
af184ded22 dedupe repeated jobs 2024-04-17 15:36:16 +08:00
Hongbo Wu
6222d567a4 add byline to the digest 2024-04-17 14:32:07 +08:00
Hongbo Wu
448e432807 runs at 1pm pst 2024-04-17 13:03:48 +08:00
Hongbo Wu
b2a55a78d3 check if feature is granted 2024-04-17 12:53:29 +08:00
Hongbo Wu
259b778aba add schedule and other parameters 2024-04-17 12:16:15 +08:00
Hongbo Wu
6a4faf6f3c add description to the digest 2024-04-17 10:04:52 +08:00
Hongbo Wu
47fd4d9663 save id, url, thumbnail and createdAt in the digest 2024-04-17 09:58:11 +08:00
Hongbo Wu
6bfb4842e2 fix idx 2024-04-16 21:27:20 +08:00
Hongbo Wu
4bc1246a47 fixing html 2024-04-16 21:00:53 +08:00
Hongbo Wu
71477b1712 implement create digest job 2024-04-16 21:00:51 +08:00
Jackson Harper
daaf493d4b Update stubbed data 2024-04-16 21:00:27 +08:00
Jackson Harper
bcddf556d5 Write speech files to redis 2024-04-16 21:00:27 +08:00
Jackson Harper
fbaa9ce663 Digest job 2024-04-16 21:00:27 +08:00
Jackson Harper
d2ea93a7dd Stub out some changes to the digest response 2024-04-16 21:00:27 +08:00
Hongbo Wu
e30fa13a2e add env to the posthog event 2024-04-16 21:00:27 +08:00
Hongbo Wu
03e497b489 use a separate queue for digest job 2024-04-16 21:00:26 +08:00
Hongbo Wu
cdb0b7525e cleanup 2024-04-16 21:00:10 +08:00
Hongbo Wu
bc955fbeeb remove stub job 2024-04-16 21:00:10 +08:00
Hongbo Wu
111dcd1562 send ranking models and summary models to posthog and log comment 2024-04-16 21:00:10 +08:00
Hongbo Wu
4342cdeb3e expire digest in one day 2024-04-16 21:00:10 +08:00
Hongbo Wu
521698646d update digest mock data 2024-04-16 21:00:10 +08:00
Hongbo Wu
a47a5a7d9c mock saving digest in redis 2024-04-16 21:00:10 +08:00
Hongbo Wu
e37bcd7408 send feedback api 2024-04-16 21:00:10 +08:00
Hongbo Wu
1edb3419f8 get digest api 2024-04-16 21:00:10 +08:00
Hongbo Wu
12aa7d4447 create digest job api 2024-04-16 21:00:10 +08:00
Hongbo Wu
b4911639c7 stub create digest job 2024-04-16 21:00:09 +08:00
Jackson Harper
be57f81bf1 Temp re-add features attribute to User 2024-04-16 13:59:04 +08:00
Hongbo Wu
8e31ac4fea fix tests 2024-04-16 12:24:20 +08:00
Hongbo Wu
d79cc1c39f wrap select in a auth transaction 2024-04-16 11:26:29 +08:00
Hongbo Wu
a9779a51e8 fix: LABEL_ALREADY_EXIST bug occured during label creation 
* comparing both user_id and label name when looking for an existing matching label
 2024-04-16 09:56:57 +08:00
Justin Maximillian Kimlim
56b87672fd
feat: increase daily upload limit 2024-04-16 00:49:26 +07:00
Hongbo Wu
9ccd1e90fe save received email as non-article by default 2024-04-15 18:54:15 +08:00
Hongbo Wu
4346746e0a save empty url for email attachment in upload_file table as it is required 2024-04-15 17:36:42 +08:00
Hongbo Wu
cbddc83829
Merge pull request #3784 from omnivore-app/feature/reply-to-email 
add reply to email API which replies Okay to the sender
 2024-04-15 15:24:32 +08:00
Hongbo Wu
e23d3d5933 update readable content instead 2024-04-12 13:26:11 +08:00
Hongbo Wu
f95642be2b remove redundant field 2024-04-11 18:47:19 +08:00
Hongbo Wu
448c78a87e add dotenv 2024-04-11 18:25:24 +08:00
Hongbo Wu
54e7c52b5e always update the state of the pdf item to succeeded/failed, even if we fail to parse the content 2024-04-11 18:07:53 +08:00
Hongbo Wu
f66d3fa36c migrate inbound-email tasks from pubsub to bullmq 2024-04-11 11:08:22 +08:00
Hongbo Wu
cd6594a2fd add logs 2024-04-11 11:08:22 +08:00
Hongbo Wu
67657773cf track the usage of the api 2024-04-11 11:08:22 +08:00
Hongbo Wu
cd6f3e6bbe enforce usage limits for the API 2024-04-11 11:08:22 +08:00
Hongbo Wu
5f239d2dcb update graceful shutdown 2024-04-11 11:08:22 +08:00
Hongbo Wu
f2f18eb12b reply only allowed text 2024-04-11 11:08:22 +08:00
Hongbo Wu
e7fc1316dd enqueue reply email job 2024-04-11 11:08:22 +08:00
Hongbo Wu
74f5b922dd update received email reply 2024-04-11 11:08:22 +08:00
Hongbo Wu
dce0b15f4f send to the reply-to address if it exists or the from address 2024-04-11 11:08:22 +08:00
Hongbo Wu
764b05fff4 save reply-to in recent emails 2024-04-11 11:08:22 +08:00
Hongbo Wu
5cc5c559ac add reply to email API which replies Okay to the sender 2024-04-11 11:08:22 +08:00
Hongbo Wu
a59c850c5f fix telegram channel 2024-04-09 19:44:16 +08:00
Hongbo Wu
e7fc2c4567 reset failed_at to null if a feed is refreshed successfully after failure 2024-04-09 18:39:17 +08:00
Hongbo Wu
77c11fb80f
Merge pull request #3767 from omnivore-app/fix/verification 
Enqueue emails
 2024-04-08 12:30:37 +08:00
Hongbo Wu
ec309f5b0d verify captcha in email-login and forgot-password api 2024-04-08 12:27:29 +08:00
Hongbo Wu
7abb396032 remove stub 2024-04-05 22:53:03 +08:00
Hongbo Wu
af140cc8b4 remove user name from email template 2024-04-05 16:59:23 +08:00
Hongbo Wu
8af2bb076c rename send mail functions 2024-04-05 16:59:20 +08:00
Hongbo Wu
c37b31dcb6 fix tests 2024-04-05 16:57:02 +08:00
Hongbo Wu
10dcb922f2 Enqueue confirmation email 2024-04-05 16:57:02 +08:00
Hongbo Wu
207241934d show label_attached instead of label_created in rules 2024-04-05 16:51:11 +08:00
Hongbo Wu
25fb9fd698 deduplicate labels when there are case-insensitive identical names 2024-04-05 16:27:17 +08:00
Hongbo Wu
6f03b214b5 allow delete internal labels 2024-04-05 15:58:39 +08:00
Hongbo Wu
9f4f1d07ca do not create label if label name case-insensitively matches existing label name 2024-04-05 15:50:01 +08:00
Hongbo Wu
906b6c1ea5 fix web 2024-04-04 18:53:10 +08:00
Hongbo Wu
58708a507e use old event format for the compatibility with the old webhooks 2024-04-04 18:22:18 +08:00
Hongbo Wu
45a5382d7c do not send bad_request error if db failure 2024-04-04 12:19:18 +08:00
Hongbo Wu
5e7a82562b resolve conflicts after rebasing 2024-04-04 12:19:17 +08:00
Hongbo Wu
b2cd43136f hide webhooks page 2024-04-04 12:19:16 +08:00
Hongbo Wu
0fab3c7904 Add highlight_updated event 2024-04-04 12:17:28 +08:00
Hongbo Wu
88a7e8d85b fix tests 2024-04-04 12:17:15 +08:00
Hongbo Wu
81f21141fe optimize ts-node-dev 2024-04-04 12:17:15 +08:00
Hongbo Wu
841a0026ff do not send page_update event for thumbnailer 2024-04-04 12:17:15 +08:00
Hongbo Wu
1b24a24f7c remove slug from highlight event 2024-04-04 12:17:15 +08:00
Hongbo Wu
d67aea6d8a deduplicate notion page by omnivore id 2024-04-04 12:17:13 +08:00
Hongbo Wu
0428da3b71 fix notion syncing error 2024-04-04 12:16:45 +08:00
Hongbo Wu
6954fffd24 fix readwise syncing error 2024-04-04 12:16:26 +08:00
Hongbo Wu
ff78a4b266 fix readwise syncing error 2024-04-04 12:16:12 +08:00
Hongbo Wu
f35ac036eb add UI for export in rules 2024-04-04 12:15:28 +08:00
Hongbo Wu
ad2e8ee002 trigger export in rules 2024-04-04 12:14:38 +08:00
Hongbo Wu
58511049a2 rename LABEL_CREATED to LABEL_ATTACHED in webhook data 2024-04-04 12:02:29 +08:00
Hongbo Wu
e2b3235046 delete unneccessary properties from webhook data 2024-04-04 12:02:29 +08:00
Hongbo Wu
88d6455222 default filter is in:all 2024-04-04 12:02:29 +08:00
Hongbo Wu
c551d9fe6b allow send data to webhook with rules 2024-04-04 12:02:29 +08:00
Hongbo Wu
07cee98ff9 allow label and highlight events to trigger rules 2024-04-04 12:02:29 +08:00
Hongbo Wu
e28dd73f9c
Merge pull request #3780 from omnivore-app/fix/notion 
sync items and highlights to user created database in notion
 2024-04-04 11:58:59 +08:00
Hongbo Wu
10cca7f05d find the title property and update its name to Title 2024-04-04 10:43:39 +08:00
Jackson Harper
c45a9d365b Also use mailjet for password changes 2024-04-03 22:25:31 +08:00
Hongbo Wu
491611b05f sync items and highlights to user created database in notion 2024-04-03 22:25:19 +08:00
Jackson Harper
3c1abb964b Also use mailjet for reset password emails 2024-04-03 22:07:28 +08:00
Hongbo Wu
f794cf707a
Merge pull request #3769 from omnivore-app/fix/notion 
export last 100 items in one job
 2024-04-03 10:22:56 +08:00
Hongbo Wu
08fb92d599 fix: site filter in rules matching both site name and url 2024-04-02 18:32:36 +08:00
Hongbo Wu
cc07d2bfbf remove comment 2024-04-02 17:56:46 +08:00
Hongbo Wu
d83dc17651 fix tests 2024-04-02 17:49:29 +08:00
Hongbo Wu
e19d6c86d8 export last 100 items in one job 2024-04-02 15:51:02 +08:00
Jackson Harper
aea354c3f2 No longer make captcha optional on web signup 2024-04-01 19:07:21 +08:00
Jackson Harper
3a59317217 Force captcha on sign up 2024-04-01 17:06:45 +08:00
Hongbo Wu
6ca092fbce use different prefix for different rate limiter 2024-04-01 13:44:10 +08:00
Hongbo Wu
a97542d4ba use redis for hourly rate limiter 2024-04-01 13:01:02 +08:00
Jackson Harper
788460c144
Merge pull request #3761 from omnivore-app/feat/signup-captcha 
Add recaptcha on the web sign up flow
 2024-04-01 12:24:39 +08:00
Jackson Harper
23f5377b62 Clean up isRecaptchaResponse function 2024-04-01 12:03:36 +08:00
Jackson Harper
0fe471da82 Dont require score on challenge recaptcha 2024-04-01 11:32:10 +08:00
Jackson Harper
822fcb1e74 Add some logging 2024-04-01 11:17:15 +08:00
Jackson Harper
9a159084c8 Update the isValidSignUp to check for URLs 2024-04-01 11:17:15 +08:00
Jackson Harper
442475d684 Make this optional for self hosters 2024-04-01 11:17:15 +08:00
Jackson Harper
7518e164a5 Utils for recaptcha 2024-04-01 11:15:19 +08:00
Hongbo Wu
5dd308d78e store the rate limit data in redis if redis is connected 2024-03-31 20:13:27 +08:00
Jackson Harper
2f6ef5e447 Return false if can not send with Mailjet 2024-03-31 19:22:56 +08:00
Jackson Harper
cfe4315fe2 Add an email fallback to Mailjet 2024-03-31 19:03:26 +08:00
Jackson Harper
200354d181 Add url present check on email sign up 2024-03-31 09:55:53 +08:00
Jackson Harper
ede99dcd33 Better matching for youtube video URLs 2024-03-30 10:42:01 +08:00
Jackson Harper
b69ba150c5 Remove debug 2024-03-29 12:28:26 +08:00
Jackson Harper
3a352a9705 Dont need to chunk this anymore 2024-03-29 12:05:32 +08:00
Jackson Harper
08ed061756 Change model for youtube transcripts 2024-03-29 11:31:27 +08:00
Hongbo Wu
f677f1167f get feature list on web 2024-03-28 15:27:37 +08:00
Hongbo Wu
cc2fe04568 add featureList to the User type which contains a list of feature type 2024-03-28 14:42:08 +08:00
Hongbo Wu
1afec1ddf3 check if feature is granted when process youtube video 2024-03-28 14:41:30 +08:00
Hongbo Wu
45417efd0c bump youtube users to 500 2024-03-28 12:15:36 +08:00
Hongbo Wu
8d8c7e8a26 fix: allow max 1000 users to use notion feature 2024-03-28 12:13:28 +08:00
Jackson Harper
b5bea17ffe Export the age of the oldest job in the queue 2024-03-28 10:25:44 +08:00
Jackson Harper
82168fe439
Merge pull request #3739 from omnivore-app/fix/youtube-queue 
Only check for summaries if the updated entity is a page item
 2024-03-28 09:40:44 +08:00
Hongbo Wu
d4ed994914
Merge pull request #3740 from omnivore-app/fix/logger 
replace logger.warn with warning
 2024-03-27 16:00:08 +08:00
Jackson Harper
38f959685f Change warn to warning 2024-03-27 15:41:15 +08:00
Hongbo Wu
92abfed2df replace logger.warn with warning 2024-03-27 15:41:13 +08:00
Jackson Harper
d23a7d56c0 Only check for summaries if the updated entity is a page item 2024-03-27 13:54:54 +08:00
Jackson Harper
5b736c1c2e
Merge pull request #3738 from omnivore-app/fix/api-youtube-logging 
Add more logging to youtube jobs
 2024-03-27 11:44:37 +08:00
Hongbo Wu
7c281359e8 fix recommendations not filtered correctly by rules 2024-03-27 11:29:47 +08:00
Jackson Harper
28222dcd56 Add more logging to youtube jobs 2024-03-27 11:25:46 +08:00
Hongbo Wu
c022cf0a22 label match wildcard 2024-03-27 10:54:24 +08:00
Hongbo Wu
3544dd1709 remove feed content from job data 2024-03-27 10:38:51 +08:00
Hongbo Wu
af78b6f4f0 append newsletter labels to avoid labels attached by rules being replaced 2024-03-27 10:37:04 +08:00
Hongbo Wu
41b90cbfd1 fix some undefined error 2024-03-26 17:53:48 +08:00
Hongbo Wu
d02d9444a9 remove content from update event too 2024-03-26 17:46:38 +08:00
Hongbo Wu
c3dba7e9da fix typo 2024-03-26 17:44:05 +08:00
Hongbo Wu
4c458c7118 fallback to search query if implicit fields are in the filter 2024-03-26 17:34:57 +08:00
Hongbo Wu
37459f64c7 remove content from job data 2024-03-26 16:49:26 +08:00
Hongbo Wu
209b08e2ff add notion to the features list 2024-03-26 12:32:01 +08:00
Hongbo Wu
f7c25fe05a do not create a new connection for the discover api 2024-03-23 10:03:06 +08:00
Hongbo Wu
282ed75a5f
Merge pull request #3698 from omnivore-app/fix/notion 
Fix: notion
 2024-03-21 14:08:16 +08:00
Hongbo Wu
445d2846b5 export last 100 items 2024-03-21 14:06:56 +08:00
Hongbo Wu
390d12eea1 add export to integration and get task api 2024-03-21 11:14:23 +08:00
Hongbo Wu
b1e3811e4f
Merge pull request #3707 from omnivore-app/fix/rule 
send labels and highlight annotations in page_create event
 2024-03-21 10:52:15 +08:00
Hongbo Wu
6385b40e38 revert batch size change 2024-03-20 23:07:46 +08:00
Hongbo Wu
3d986a9036 send labels and highlight annotations in page_create event 2024-03-20 22:33:24 +08:00
Jackson Harper
59cc954618 Add 12hrs option, fix linting issue 2024-03-20 15:36:53 +08:00
Jackson Harper
82ef57a36a Add last24hrs query for time based search queries to get recently saved/changed items 2024-03-20 15:15:18 +08:00
Jackson Harper
014ccb1eae Set published date from youtube data 
its not clear how consistent the formatting is on these dates,
the one i tested seems like it was a UTC formatted dd M, yyyy string
 2024-03-20 13:10:41 +08:00
Hongbo Wu
689eeaeacd disable sending pubsub events for rss feed temporarily to resolve conflicts 2024-03-20 10:46:13 +08:00
Hongbo Wu
2e374557bf send labels in the page_created events 2024-03-20 10:36:36 +08:00
Hongbo Wu
abf9706539 resolve conflicts after rebasing 2024-03-20 10:36:21 +08:00
Hongbo Wu
7f224cc1e1 running text search on all text fields 2024-03-20 10:36:21 +08:00
Hongbo Wu
edeafc9a71 fix bug 2024-03-20 10:36:21 +08:00
Hongbo Wu
de28a9ad5f fix bug 2024-03-20 10:36:21 +08:00
Hongbo Wu
f1eff130af filter data locally with liqe 2024-03-20 10:36:21 +08:00
Hongbo Wu
e87e6dd102 filter data locally 2024-03-20 10:36:21 +08:00
Hongbo Wu
a49366b4cc remove unnecessary fields from item events 2024-03-20 10:36:19 +08:00
Hongbo Wu
208a5895ef remove unnecessary fields from item events 2024-03-20 10:35:58 +08:00
Hongbo Wu
9af4235233 skip failed to parse rules and show the timestamp in the UI 2024-03-20 10:35:58 +08:00
Hongbo Wu
c05721c87e validate the filter before creating rules 2024-03-20 10:35:57 +08:00
Hongbo Wu
76f3321680 revalidate integration once updated 2024-03-20 10:32:32 +08:00
Hongbo Wu
bf59fbf26e add integration api 2024-03-20 10:32:31 +08:00
Hongbo Wu
7e04b25497
Merge pull request #3696 from omnivore-app/fix/thumbnail 
fix/thumbnail
 2024-03-20 10:26:34 +08:00
Hongbo Wu
315c625482 fix title not fetched correctly when using preview content in rss feeds 2024-03-19 18:07:27 +08:00
Jackson Harper
e47e169402
Merge branch 'main' into discover-no-ts-changes 2024-03-19 15:50:51 +08:00
Hongbo Wu
ad32aa82ce Export highlight notes as children of highlights block in notion 2024-03-19 11:17:42 +08:00
Jackson Harper
21d4efc2b5 Allow opting into the YouTube transcript feature 2024-03-18 13:56:37 +08:00
Jackson Harper
4e50c51598
Merge pull request #3661 from omnivore-app/feat/job-youtube-transcript-generator 
Pull metadata from YouTube for library items
 2024-03-17 09:40:28 +08:00
Jackson Harper
596ab5a7aa Feature flag the transcripts 2024-03-15 20:28:02 +08:00
Jackson Harper
b87073b1fb Add youtube-transcripts beta feature 2024-03-15 20:25:44 +08:00
Jackson Harper
d3aa671200 Prettier 2024-03-15 10:44:40 +08:00
Jackson Harper
fdc8055b79 Prettier 2024-03-15 10:43:16 +08:00
Hongbo Wu
17e66aa010 append new highlights only 2024-03-14 22:13:27 +08:00
Hongbo Wu
1085dcc824 read settings 2024-03-14 21:36:47 +08:00
Hongbo Wu
69e74d432d append highlights to the existing page 2024-03-14 19:50:26 +08:00
Jackson Harper
f2d23626a5 Improve error logging / fix linting 2024-03-14 17:39:13 +08:00
Jackson Harper
dc6c047aec Add GCS cache 2024-03-14 16:12:06 +08:00
Jackson Harper
629c044273 Queue the transcript processing as a separate job 
Handle YouTube in two steps, first get metadata then
get the transcript.
 2024-03-14 15:09:53 +08:00
Hongbo Wu
1ea09e20bd fix: subscription not updated correctly after rss feed refreshed 
* sort the shallow copy of the userIds array so the original array not mutated
 2024-03-14 12:56:42 +08:00
Jackson Harper
fef28d1c6a Linting fix 2024-03-14 12:48:56 +08:00
Jackson Harper
3ee6787e39 Improve transcript generation 2024-03-14 12:48:56 +08:00
Jackson Harper
39dfa920b5 More fixes to youtube processor 2024-03-14 12:48:56 +08:00
Jackson Harper
6bcdbfa8f0 Linting 2024-03-14 12:48:56 +08:00
Jackson Harper
7c3d15e31a Add some scrolling on youtube videos 2024-03-14 12:48:56 +08:00
Jackson Harper
2dbd16a61e Pull duration and description from YouTube metadata 2024-03-14 12:48:56 +08:00
Jackson Harper
f7225b298a Rebase 2024-03-14 12:48:56 +08:00
Hongbo Wu
81e489021e add popular reads to the continue reading section of new user library 2024-03-14 11:14:37 +08:00