prehistoric-systems/omnivore
1
0
Fork 0
mirror of https://github.com/omnivore-app/omnivore.git synced 2026-03-11 08:54:26 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
12057 commits 201 branches 31 tags 432 MiB
Commit graph

2536 commits

Author SHA1 Message Date
Hongbo Wu
6f12b5e406 reduce bulk action batch size to 50 2024-07-25 10:54:21 +08:00
Hongbo Wu
dee421d3c8 add hour rate limiter middleware to the api router and allows max 600 requests per hour 2024-07-24 16:24:49 +08:00
Hongbo Wu
6776cdf1cd check if there are more than five items "created" within a minute instead of "saved" 2024-07-24 15:59:22 +08:00
Hongbo Wu
95657e3d3e queue email sending jobs in exporter and importer 2024-07-21 11:11:29 +08:00
Hongbo Wu
3f76431c47 use new gpt-4o-mini openai model 2024-07-19 11:45:38 +08:00
Hongbo Wu
b5fe0cb279 remove all the commas in the raw transcript which are added by youtubei 2024-07-18 12:10:45 +08:00
Hongbo Wu
df3c16381a fix: revert home redirect path to /home 2024-07-16 12:47:41 +08:00
Jackson Harper
b6d0c2723d
Merge pull request #4188 from debanjandhar12/main 
Add Amplenote origins to cors config
 2024-07-12 13:54:34 +08:00
Hongbo Wu
3d791a7866
Merge pull request #4155 from omnivore-app/fingerprint 
inject fingerprint
 2024-07-11 14:24:07 +08:00
Hongbo Wu
1965f0ac19 fix tests 2024-07-11 13:38:22 +08:00
Hongbo Wu
d36d9cdb57 fix: update highlight 2024-07-11 13:14:01 +08:00
Hongbo Wu
225eefd73b fix: readwise export not working for PAGE and LABEL events 2024-07-11 13:13:50 +08:00
Hongbo Wu
2e322955f6 fix: title not exported when item is not in notion and gets exported by attaching labels 2024-07-11 12:24:25 +08:00
Hongbo Wu
4e388de7ed fix: export highlight in yellow in notion if color is null 2024-07-11 11:41:20 +08:00
debanjandhar12
af48aa009e Add Amplenote origins to cors config 2024-07-10 20:00:39 +05:30
Jackson Harper
91ca8acfe5
Merge pull request #4186 from omnivore-app/fix/importer 
expose createdAt for User
 2024-07-10 21:01:27 +08:00
Hongbo Wu
a1ae03f65f expose createdAt for User 2024-07-10 18:40:01 +08:00
Hongbo Wu
8e5439fa7b generate random user-agent when fetching rss feed 2024-07-10 17:59:14 +08:00
Hongbo Wu
1aaa4202fc move items which have less than 250 words to quick links section 2024-07-10 10:54:36 +08:00
Hongbo Wu
8ac27171c7 fix: include just added items in top picks 2024-07-10 10:51:50 +08:00
Jackson Harper
9134a3325f Set default home redirect path 2024-07-09 15:29:34 +08:00
Jackson Harper
180efb4846 Set Android client 
This checks user-agent before X-OmnivoreClient so the more explicit
X-OmnivoreClient overrides the derived client being set from user
agent.
 2024-07-05 20:15:31 +08:00
Hongbo Wu
eee38fcbb9
Merge pull request #4153 from omnivore-app/fix/intercom 
have separate intercom secret in env var and return the hash based on the client
 2024-07-05 18:00:05 +08:00
Hongbo Wu
76ebfbd305 create separate intercom secret and return the hash based on the client 2024-07-05 17:03:27 +08:00
Jackson Harper
e2edaa678b
Merge pull request #4149 from omnivore-app/feat/ml-score-v3 
V3 scoring model, add prom monitoring
 2024-07-05 16:58:00 +08:00
Jackson Harper
59852a1565 Throw if we cant sign the request 2024-07-05 16:43:29 +08:00
Jackson Harper
6a201018b1 Fix promise 2024-07-05 15:20:45 +08:00
Jackson Harper
a7ca844220 Add authorization to digest-score 2024-07-05 14:59:16 +08:00
Hongbo Wu
e3ffafd623
Merge pull request #4142 from omnivore-app/fix/score-client 
increase score api timeout to 20 seconds
 2024-07-04 18:01:47 +08:00
Hongbo Wu
e9dcdc6dd1 fix: word count in CJK languages 2024-07-04 12:28:01 +08:00
Hongbo Wu
56032b0765 increase score api timeout to 20 seconds 2024-07-04 10:32:09 +08:00
Jackson Harper
b23e257206 Add original_url to score service calls 2024-07-02 13:58:15 +08:00
Hongbo Wu
ee70f6603a return not_found if database is not found in notion 2024-07-01 17:42:19 +08:00
Hongbo Wu
67af18a105 fix: returns a stub score (0) in case score api throws an error 2024-07-01 15:27:16 +08:00
Hongbo Wu
70829a6b88 remove initial observation 2024-07-01 12:08:56 +08:00
Hongbo Wu
ad88cbaea8 observe failed digest-score api call lantency in prometheus too 2024-07-01 12:05:48 +08:00
Hongbo Wu
550cef241c observe digest-score api lantency in prometheus 2024-07-01 12:01:42 +08:00
Hongbo Wu
9f7065c5c3 do not update library item score if 0 2024-07-01 11:51:57 +08:00
Hongbo Wu
6bb11ce99b fix: returns a stub score (0) in case score api throws an error 2024-07-01 11:02:42 +08:00
Hongbo Wu
5d07e554e2 return author name only 2024-07-01 10:49:09 +08:00
Hongbo Wu
fa3e6fe341 make libraryItemIds not null 2024-07-01 10:49:09 +08:00
Hongbo Wu
72a477c25a add delete post graphql api and tests 2024-07-01 10:49:09 +08:00
Hongbo Wu
16f67c854c add update post graphql api and tests 2024-07-01 10:49:09 +08:00
Hongbo Wu
9caad12f09 add create post graphql api and tests 2024-07-01 10:49:09 +08:00
Hongbo Wu
12db8b6a7d add get post graphql api and tests 2024-07-01 10:49:09 +08:00
Hongbo Wu
81b0fc19a2 tidy 2024-07-01 10:49:08 +08:00
Hongbo Wu
091af055c0 add tests 2024-07-01 10:11:32 +08:00
Hongbo Wu
e9b945b375 add tests 2024-07-01 10:11:32 +08:00
Hongbo Wu
3fb146e864 list posts api 2024-07-01 10:11:32 +08:00
Hongbo Wu
c02f2d5036 create post table and entity 2024-07-01 10:11:32 +08:00
Hongbo Wu
0dada2d708 fix: digest score api timeout = 5 seconds 2024-06-28 16:38:00 +08:00
Hongbo Wu
a7ba02e063
Merge pull request #4117 from omnivore-app/fix/use-read-replica 
fix/use read replica for RLS enabled transactions
 2024-06-28 15:13:26 +08:00
Hongbo Wu
a06f1a243c fix tests 2024-06-28 12:37:45 +08:00
Hongbo Wu
a3ba8573b7 fix: validate ownership of library item before assigning labels to the item 2024-06-28 11:53:20 +08:00
Hongbo Wu
3ad826392e use digest-score service 2024-06-27 20:55:04 +08:00
Hongbo Wu
822f249fb1 cont 2024-06-27 19:10:50 +08:00
Hongbo Wu
2ed480bb49 use read replica for selecting library items, labels and highlights 2024-06-27 19:02:15 +08:00
Hongbo Wu
fb3d2eada6 use primary and replica instead of master and slave 2024-06-27 18:42:42 +08:00
Hongbo Wu
0830f0b312 allow defining replica mode in the auth transaction 2024-06-27 18:38:43 +08:00
Hongbo Wu
17eb5efa37 resolve conflicts 2024-06-27 17:16:17 +08:00
Hongbo Wu
45986184c4 tidy 2024-06-27 17:12:45 +08:00
Hongbo Wu
1168d12018 use read replica for raw query 2024-06-27 17:10:58 +08:00
Hongbo Wu
84953198fe
Merge pull request #4114 from omnivore-app/fix/read-replica 
fix/read replica
 2024-06-26 17:38:53 +08:00
Hongbo Wu
329dd13ed2 return empty array if no candidates for home feed selection 2024-06-26 17:19:08 +08:00
Hongbo Wu
a6ab96155d by default use info level for loggers 2024-06-26 16:19:13 +08:00
Hongbo Wu
818ad5a0bb log isReplicated for each query 2024-06-26 16:11:52 +08:00
Hongbo Wu
bc6a09798b fix read replica not being used 2024-06-26 16:11:34 +08:00
Hongbo Wu
57e0002cae rename slave to replica in db config 2024-06-26 11:34:45 +08:00
Hongbo Wu
9f16e5fede add "canMove" field to the HomeItem type in graphql api which indicates if the item is in following folder 2024-06-25 16:40:12 +08:00
Hongbo Wu
a1cd611fa7 use read replica for "read" 2024-06-21 14:39:40 +08:00
Hongbo Wu
b0d6876e56 add replication config 2024-06-21 13:31:50 +08:00
Hongbo Wu
14a375db83 use real scoring service 2024-06-21 12:01:34 +08:00
Hongbo Wu
977a1a90e5 Skip scoring RSS items for now 2024-06-21 12:00:53 +08:00
Hongbo Wu
8c84fc58b8 add more subscription features to the score api payload 2024-06-20 17:36:57 +08:00
Hongbo Wu
40182421f7 show archived highlights 2024-06-20 16:31:05 +08:00
Hongbo Wu
2e26a04cd0 remove redundant context type 2024-06-20 15:47:52 +08:00
Jackson Harper
4ab0497034 Get uid from claims 2024-06-19 18:09:00 +08:00
Jackson Harper
54f608d23c Use context uid 2024-06-19 17:53:08 +08:00
Jackson Harper
e2e0774e7e Add user field to the ArticleSavingRequest resolver 2024-06-19 17:44:44 +08:00
Hongbo Wu
6df8c71035 reduce bullmq worker concurrency to 2 2024-06-18 17:28:19 +08:00
Hongbo Wu
2ed63af9a4 stub score client 2024-06-18 12:10:03 +08:00
Hongbo Wu
47c6488834 do not update library item score in database for now 2024-06-18 11:25:44 +08:00
Hongbo Wu
a26e8a3972
Merge pull request #4041 from omnivore-app/feature/folder-policy-api 
feat: folder expiration policy api
 2024-06-14 14:44:21 +08:00
Hongbo Wu
c0947ddd48 fix batch delete query does not use index 2024-06-14 12:21:20 +08:00
Hongbo Wu
aea1650626 do not show move/archived/deleted item in home feed 2024-06-14 12:15:17 +08:00
Hongbo Wu
be8185d79c
Merge pull request #4059 from omnivore-app/fix/rss 
fix: fetch old items for existing subscribed feeds
 2024-06-14 09:04:37 +08:00
Hongbo Wu
31f04344fc resolve rebase conflicts 2024-06-13 18:41:35 +08:00
Hongbo Wu
88e3d648c8 add a REST API to trigger the expire folder job 2024-06-13 18:36:33 +08:00
Hongbo Wu
6f496b9336 add folder expiration jobs 2024-06-13 18:30:27 +08:00
Hongbo Wu
12fa6ca0b4 add delete folder policy api 2024-06-13 18:29:21 +08:00
Hongbo Wu
6138e078b1 add update folder policy api 2024-06-13 18:29:21 +08:00
Hongbo Wu
2d757a4896 add create folder policy api 2024-06-13 18:29:21 +08:00
Hongbo Wu
7940c3f4ea add list folder policy api 2024-06-13 18:29:21 +08:00
Hongbo Wu
25ca4c7791 create folder_policy table 2024-06-13 18:29:21 +08:00
Hongbo Wu
e3d70fc27d
Merge pull request #4052 from omnivore-app/feature/prune-trash-items 
feature/prune trash items
 2024-06-13 17:58:18 +08:00
Hongbo Wu
9ef91ff1ca add an REST API to trigger the prune trash job 2024-06-13 14:48:14 +08:00
Hongbo Wu
c82efb31d3 fetch old items for existing subscribed feeds 2024-06-13 14:15:19 +08:00
Jackson Harper
c7b1794591 Implement more of Recently Added section 2024-06-12 21:00:14 +08:00
Jackson Harper
e9f3009111
Merge pull request #4040 from omnivore-app/feat/web-layout-navigation 
Update layout navigation with left menu
 2024-06-12 20:02:42 +08:00
Hongbo Wu
71d07caf94 run prune trash job at 3am UTC daily 2024-06-12 19:27:42 +08:00
Hongbo Wu
46aa405552 remove timeout in highlight generator 2024-06-12 19:13:09 +08:00
Hongbo Wu
f29592d671 worker can concurrently process 10 jobs 2024-06-12 18:40:08 +08:00
Hongbo Wu
a580fd7ebf catch sync reading progress query error 2024-06-12 18:40:07 +08:00
Hongbo Wu
fc5b152c43 update job name 2024-06-12 18:39:27 +08:00
Hongbo Wu
755f483ba9 create a function to batch delete items in trash 2024-06-12 18:38:14 +08:00
Jackson Harper
5f817bc336 Null handler as we know URL must be set for RSS subscriptions 2024-06-12 14:00:45 +08:00
Hongbo Wu
a5295feb91 remove console timer logs 2024-06-12 13:37:23 +08:00
Hongbo Wu
57695c370c monitor each job latency in prometheus 2024-06-12 13:35:04 +08:00
Hongbo Wu
f052ab55a0 monitor home feed job latency in prometheus 2024-06-12 13:01:36 +08:00
Hongbo Wu
bc7e710835 open youtube transcript to public 2024-06-11 10:55:56 +08:00
Jackson Harper
3f6e7c924a Work on new left menu w/shortcuts 2024-06-10 17:34:59 +08:00
Hongbo Wu
470413b704 add basic highlights web UI 2024-06-08 10:50:44 +08:00
Hongbo Wu
c826e034ab
Merge pull request #4034 from omnivore-app/fix/home-item-thumbnail 
fix: home item thumbnail not using proxy url
 2024-06-07 13:41:15 +08:00
Hongbo Wu
e05985b2e4 fix typo 2024-06-07 13:40:54 +08:00
Hongbo Wu
0db1108675 replace continuous spaces and new lines with space in preview content 2024-06-07 13:29:38 +08:00
Hongbo Wu
76383d8a33 feat: generate preview content if excerpt less than 180 chars 2024-06-06 19:08:41 +08:00
Hongbo Wu
c48013ed4b fix: home item thumbnail not using proxy url 2024-06-06 17:50:18 +08:00
Hongbo Wu
70bc136d15 batch get labels from highlight id 2024-06-06 17:26:01 +08:00
Hongbo Wu
24457d9036 search highlights by labels 2024-06-06 16:12:07 +08:00
Hongbo Wu
51efff8b8e allow returning library item information in highlight 2024-06-06 15:23:56 +08:00
Hongbo Wu
9259a9cfe3 fix tests 2024-06-06 14:53:14 +08:00
Hongbo Wu
166b324d63 add tests 2024-06-06 13:26:13 +08:00
Hongbo Wu
c620ef38a2 clean up helpers 2024-06-06 12:08:15 +08:00
Hongbo Wu
0013150c26 feat: highlights api 2024-06-05 20:44:18 +08:00
Hongbo Wu
1df84f7a42
Merge pull request #4022 from omnivore-app/fix/notion 
fix: always sync Last Updated in Notion
 2024-06-03 13:01:49 +08:00
Hongbo Wu
e1236dbe29 always sync Last Updated 2024-06-02 09:56:17 +08:00
Hongbo Wu
8eac9375a2 feat: add a hidden section in home feed and load on demand 2024-05-31 16:34:32 +08:00
Hongbo Wu
e932777347 fix ranking order 2024-05-31 12:58:01 +08:00
Hongbo Wu
a6d27bea8d rename for you to top picks 2024-05-31 12:23:11 +08:00
Hongbo Wu
aa4e091c4b exclude just added items from for you and quick links section 2024-05-31 12:00:30 +08:00
Hongbo Wu
1af6cfa549 rename long section to for you 2024-05-31 11:51:30 +08:00
Hongbo Wu
633a022825 add client side pagination 2024-05-30 17:36:18 +08:00
Hongbo Wu
d349c1867f add section title 2024-05-30 15:48:57 +08:00
Hongbo Wu
8dfd876392 limit to 5 just added items 2024-05-30 15:48:28 +08:00
Hongbo Wu
bc4951f376 do not fail score job if failed to enqueue updating home job 2024-05-30 15:39:04 +08:00
Hongbo Wu
3c27125645 trigger update home job once a new item is scored 2024-05-30 14:34:08 +08:00
Hongbo Wu
4bb33da27f add just added section to home 2024-05-30 13:43:12 +08:00
Hongbo Wu
5739ec2843 only score candidates without scores 2024-05-29 18:47:22 +08:00
Hongbo Wu
84a6530743 log job error 2024-05-29 18:29:48 +08:00
Hongbo Wu
c7de24e9f8 do not rank candidates and just return them all if less than 10 candidates 2024-05-29 18:22:16 +08:00
Hongbo Wu
d1d7f93fc3 time each steps in the update home job 2024-05-29 17:58:28 +08:00
Hongbo Wu
69dbbd7635 add score api url to env var 2024-05-29 17:45:51 +08:00
Hongbo Wu
b7e6888395 remove debugging logs 2024-05-29 17:42:30 +08:00
Hongbo Wu
69c555b8c3
Merge pull request #4005 from omnivore-app/feature/refresh-home-api 
feature/refresh home api
 2024-05-29 17:37:41 +08:00
Hongbo Wu
b6811ba957 delete home cache before recreating 2024-05-29 17:33:01 +08:00
Hongbo Wu
91c2ae8d97 add refresh home api 2024-05-29 17:24:37 +08:00
Hongbo Wu
a9ebcd33e4 add mark as seen to the bulk action 2024-05-29 17:07:19 +08:00
Hongbo Wu
f496664025 remove expired items before remove extra items 2024-05-29 16:42:16 +08:00
Jackson Harper
7466501c95 Parse the single score from the digest-score service 2024-05-29 16:40:44 +08:00
Hongbo Wu
9280703047
Merge pull request #4003 from omnivore-app/fix/add-score-to-item 
fix/add score to item
 2024-05-29 15:19:43 +08:00