prehistoric-systems/omnivore
1
0
Fork 0
mirror of https://github.com/omnivore-app/omnivore.git synced 2026-03-11 08:54:26 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
12057 commits 201 branches 31 tags 432 MiB
Commit graph

2536 commits

Author SHA1 Message Date
Hongbo Wu
5c9816b5b8 store verification token in redis with exp and destroy after use 2024-08-22 18:04:18 +08:00
Hongbo Wu
47cea15480 keep up to 1 hour complete tasks and up to 1 day failed tasks 2024-08-22 15:27:59 +08:00
Hongbo Wu
3cd3817569 update the env var name 2024-08-22 12:27:26 +08:00
Hongbo Wu
fe87d23a91 use cloud task for fetching content of rss items if USE_CONTENT_FETCH_QUEUE is not set to true in the environment 2024-08-22 12:24:41 +08:00
Hongbo Wu
aa2caca944 do not cache content from https://jacksonh.org 2024-08-21 18:47:23 +08:00
Hongbo Wu
0366c426bc process up to 2 jobs concurrently 2024-08-21 17:58:58 +08:00
Hongbo Wu
6de285432d fix importer status not updated if failed to fetch content 2024-08-21 16:53:50 +08:00
Hongbo Wu
5bd272dde0 use one queue with different priority for fetching content of rss feed item or saved url 2024-08-21 12:24:35 +08:00
Hongbo Wu
e41ccf7a8f deduplicate content fetch jobs 2024-08-21 12:24:35 +08:00
Hongbo Wu
08fbb8aebf use different queues for fast,slow and rss content fetch jobs 2024-08-21 12:24:35 +08:00
Hongbo Wu
87b4ec503e enqueue content-fetch task to the queue 2024-08-21 12:24:35 +08:00
Jackson Harper
deaa0e8856 Use an upsert for the default shortcuts 2024-08-20 16:41:04 +08:00
Hongbo Wu
160d690589 fix: do not select content from database for update since api 2024-08-19 10:21:24 +08:00
Hongbo Wu
19ed5501f6 fix: select readable_content from library_item table if content is requested by search api 2024-08-19 10:09:42 +08:00
Hongbo Wu
626c456364 fix: select readable_content from library_item table if content is requested by update since api 2024-08-19 10:00:38 +08:00
Hongbo Wu
26c7e95a00 update folder 2024-08-16 16:50:46 +08:00
Hongbo Wu
accab510a7 if only rank orders are present, add saved at order 2024-08-16 15:15:11 +08:00
Hongbo Wu
173691048b perf: remove additional order by clause to use the correct order index 2024-08-15 22:10:36 +08:00
Hongbo Wu
32097de62f perf: reduce number of column returned by search query 2024-08-15 21:24:57 +08:00
Hongbo Wu
3a07b21ec7 perf: stop serving content in search and update since api 2024-08-15 21:09:40 +08:00
Hongbo Wu
efb1feb86a perf: compress response body 2024-08-15 20:44:08 +08:00
Jackson Harper
7c7a55e475 Stop support for old plugin clients 2024-08-14 19:31:23 +08:00
Hongbo Wu
cff44b83a8 fix: total count cache should be fixed for pagination 2024-08-14 16:10:41 +08:00
Hongbo Wu
c9aa90b3ae more debugging logs 2024-08-14 13:04:29 +08:00
Hongbo Wu
8b9ab287d7 add debugging logs 2024-08-14 13:00:53 +08:00
Hongbo Wu
4bf3e4ba0f delete count cache after bulk action or empty trash 2024-08-14 12:59:52 +08:00
Hongbo Wu
4fcf025611 update cache when user is updated 2024-08-14 12:24:35 +08:00
Hongbo Wu
4c992f74ec
Merge pull request #4260 from omnivore-app/perf/cache-api-response 
perf/cache api response
 2024-08-14 11:32:19 +08:00
Jackson Harper
e0b1a01f8f Set a max age on CORs requests 2024-08-14 10:19:17 +08:00
Hongbo Wu
45fdc0d474 cache shortcuts for 600 seconds 2024-08-14 10:18:11 +08:00
Hongbo Wu
e05ff942b1 cache features for 600 seconds 2024-08-14 10:09:37 +08:00
Hongbo Wu
bcdff19a3d cache total count of library items in page info only 2024-08-14 09:40:59 +08:00
Hongbo Wu
81ff5a05f9 extend count cache ttl to 600 seconds 2024-08-14 09:40:59 +08:00
Hongbo Wu
d00a76ca3f hash args in the cache key 2024-08-14 09:40:59 +08:00
Hongbo Wu
c2ec95845e request count of library items only when needed 2024-08-14 09:40:59 +08:00
Jackson Harper
6f6e42eeda Fix typo 2024-08-13 23:51:59 +08:00
Jackson Harper
588f7d0fdc Use a separate section for search so we can better control the folder 2024-08-13 23:38:18 +08:00
Hongbo Wu
de62dcfe43 add highlightsCount to the searchItem graphql type 2024-08-13 17:59:28 +08:00
Hongbo Wu
3fe22e0f8e
Merge pull request #4215 from omnivore-app/feature/exporter 
create a cloud run service for the exporter
 2024-08-13 17:20:44 +08:00
Hongbo Wu
d633b8c1c3 cache total count of library items in 60 seconds 2024-08-13 10:52:28 +08:00
Hongbo Wu
ee039833ac remove hidden home section resolver 2024-08-08 13:06:51 +08:00
Hongbo Wu
41b6a27601 disable digest-score and update-home jobs 2024-08-08 12:54:33 +08:00
Hongbo Wu
b5629e5f9f remove home resolvers 2024-08-08 12:53:28 +08:00
Hongbo Wu
4de6cfc1f8 fix: return markdown content for the old clients when highlighted markdown is requested 2024-08-07 09:34:35 +08:00
Hongbo Wu
64cf10205b fix tests 2024-08-06 14:44:15 +08:00
Hongbo Wu
322f736fe0 stop storing original html in the database 2024-07-31 19:14:38 +08:00
Hongbo Wu
bdae03ef95
Merge pull request #4221 from omnivore-app/fix/content-fetch-error 
improve content-fetch
 2024-07-25 13:40:14 +08:00
Hongbo Wu
156feb8e17 rate limit save url action in redis: max 5 save urls in a minute 2024-07-25 13:08:38 +08:00
Hongbo Wu
ee5e66d9b3
Merge pull request #4075 from omnivore-app/feature/posts 
feat: posts api
 2024-07-25 11:42:35 +08:00
Hongbo Wu
465e5dc7d1 Add an index of columns user_id and created_at to the library_item table for counting query 2024-07-25 11:18:52 +08:00
Hongbo Wu
6f12b5e406 reduce bulk action batch size to 50 2024-07-25 10:54:21 +08:00
Hongbo Wu
dee421d3c8 add hour rate limiter middleware to the api router and allows max 600 requests per hour 2024-07-24 16:24:49 +08:00
Hongbo Wu
6776cdf1cd check if there are more than five items "created" within a minute instead of "saved" 2024-07-24 15:59:22 +08:00
Hongbo Wu
95657e3d3e queue email sending jobs in exporter and importer 2024-07-21 11:11:29 +08:00
Hongbo Wu
3f76431c47 use new gpt-4o-mini openai model 2024-07-19 11:45:38 +08:00
Hongbo Wu
b5fe0cb279 remove all the commas in the raw transcript which are added by youtubei 2024-07-18 12:10:45 +08:00
Hongbo Wu
df3c16381a fix: revert home redirect path to /home 2024-07-16 12:47:41 +08:00
Jackson Harper
b6d0c2723d
Merge pull request #4188 from debanjandhar12/main 
Add Amplenote origins to cors config
 2024-07-12 13:54:34 +08:00
Hongbo Wu
3d791a7866
Merge pull request #4155 from omnivore-app/fingerprint 
inject fingerprint
 2024-07-11 14:24:07 +08:00
Hongbo Wu
1965f0ac19 fix tests 2024-07-11 13:38:22 +08:00
Hongbo Wu
d36d9cdb57 fix: update highlight 2024-07-11 13:14:01 +08:00
Hongbo Wu
225eefd73b fix: readwise export not working for PAGE and LABEL events 2024-07-11 13:13:50 +08:00
Hongbo Wu
2e322955f6 fix: title not exported when item is not in notion and gets exported by attaching labels 2024-07-11 12:24:25 +08:00
Hongbo Wu
4e388de7ed fix: export highlight in yellow in notion if color is null 2024-07-11 11:41:20 +08:00
debanjandhar12
af48aa009e Add Amplenote origins to cors config 2024-07-10 20:00:39 +05:30
Jackson Harper
91ca8acfe5
Merge pull request #4186 from omnivore-app/fix/importer 
expose createdAt for User
 2024-07-10 21:01:27 +08:00
Hongbo Wu
a1ae03f65f expose createdAt for User 2024-07-10 18:40:01 +08:00
Hongbo Wu
8e5439fa7b generate random user-agent when fetching rss feed 2024-07-10 17:59:14 +08:00
Hongbo Wu
1aaa4202fc move items which have less than 250 words to quick links section 2024-07-10 10:54:36 +08:00
Hongbo Wu
8ac27171c7 fix: include just added items in top picks 2024-07-10 10:51:50 +08:00
Jackson Harper
9134a3325f Set default home redirect path 2024-07-09 15:29:34 +08:00
Jackson Harper
180efb4846 Set Android client 
This checks user-agent before X-OmnivoreClient so the more explicit
X-OmnivoreClient overrides the derived client being set from user
agent.
 2024-07-05 20:15:31 +08:00
Hongbo Wu
eee38fcbb9
Merge pull request #4153 from omnivore-app/fix/intercom 
have separate intercom secret in env var and return the hash based on the client
 2024-07-05 18:00:05 +08:00
Hongbo Wu
76ebfbd305 create separate intercom secret and return the hash based on the client 2024-07-05 17:03:27 +08:00
Jackson Harper
e2edaa678b
Merge pull request #4149 from omnivore-app/feat/ml-score-v3 
V3 scoring model, add prom monitoring
 2024-07-05 16:58:00 +08:00
Jackson Harper
59852a1565 Throw if we cant sign the request 2024-07-05 16:43:29 +08:00
Jackson Harper
6a201018b1 Fix promise 2024-07-05 15:20:45 +08:00
Jackson Harper
a7ca844220 Add authorization to digest-score 2024-07-05 14:59:16 +08:00
Hongbo Wu
e3ffafd623
Merge pull request #4142 from omnivore-app/fix/score-client 
increase score api timeout to 20 seconds
 2024-07-04 18:01:47 +08:00
Hongbo Wu
e9dcdc6dd1 fix: word count in CJK languages 2024-07-04 12:28:01 +08:00
Hongbo Wu
56032b0765 increase score api timeout to 20 seconds 2024-07-04 10:32:09 +08:00
Jackson Harper
b23e257206 Add original_url to score service calls 2024-07-02 13:58:15 +08:00
Hongbo Wu
ee70f6603a return not_found if database is not found in notion 2024-07-01 17:42:19 +08:00
Hongbo Wu
67af18a105 fix: returns a stub score (0) in case score api throws an error 2024-07-01 15:27:16 +08:00
Hongbo Wu
70829a6b88 remove initial observation 2024-07-01 12:08:56 +08:00
Hongbo Wu
ad88cbaea8 observe failed digest-score api call lantency in prometheus too 2024-07-01 12:05:48 +08:00
Hongbo Wu
550cef241c observe digest-score api lantency in prometheus 2024-07-01 12:01:42 +08:00
Hongbo Wu
9f7065c5c3 do not update library item score if 0 2024-07-01 11:51:57 +08:00
Hongbo Wu
6bb11ce99b fix: returns a stub score (0) in case score api throws an error 2024-07-01 11:02:42 +08:00
Hongbo Wu
5d07e554e2 return author name only 2024-07-01 10:49:09 +08:00
Hongbo Wu
fa3e6fe341 make libraryItemIds not null 2024-07-01 10:49:09 +08:00
Hongbo Wu
72a477c25a add delete post graphql api and tests 2024-07-01 10:49:09 +08:00
Hongbo Wu
16f67c854c add update post graphql api and tests 2024-07-01 10:49:09 +08:00
Hongbo Wu
9caad12f09 add create post graphql api and tests 2024-07-01 10:49:09 +08:00
Hongbo Wu
12db8b6a7d add get post graphql api and tests 2024-07-01 10:49:09 +08:00
Hongbo Wu
81b0fc19a2 tidy 2024-07-01 10:49:08 +08:00
Hongbo Wu
091af055c0 add tests 2024-07-01 10:11:32 +08:00
Hongbo Wu
e9b945b375 add tests 2024-07-01 10:11:32 +08:00
Hongbo Wu
3fb146e864 list posts api 2024-07-01 10:11:32 +08:00
Hongbo Wu
c02f2d5036 create post table and entity 2024-07-01 10:11:32 +08:00