prehistoric-systems/omnivore
1
0
Fork 0
mirror of https://github.com/omnivore-app/omnivore.git synced 2026-03-11 08:54:26 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
12057 commits 201 branches 31 tags 432 MiB
Commit graph

2536 commits

Author SHA1 Message Date
Hongbo Wu
a3d3f4daf4 feat: allow logical operator in search query 2023-12-05 10:58:18 +08:00
Hongbo Wu
120fe19dab fix typo 2023-12-04 18:53:46 +08:00
Hongbo Wu
a116833659 feat: allow moving items to a folder in a bulk 2023-12-04 18:10:14 +08:00
Hongbo Wu
37beab64c6
Merge pull request #3198 from omnivore-app/fix/move-to-folder-api 
fix moveToFolder API error by skipping fetching content again
 2023-12-04 16:10:09 +08:00
Hongbo Wu
767629b8c7 replace articleSavingRequest with success (boolean value) as a response 2023-12-04 15:54:14 +08:00
Hongbo Wu
824b8187af use system token in integration handler 2023-12-04 15:38:08 +08:00
Hongbo Wu
bafcdb2fee skip rate limiter for system requests 2023-12-04 14:56:26 +08:00
Hongbo Wu
e33c4ad685 fix moveToFolder API error by skipping fetching content again 2023-12-04 14:29:54 +08:00
Hongbo Wu
fe86085245 update comment 2023-12-01 12:19:05 +08:00
Hongbo Wu
706ff94cd1 allow use:folders in search string for new clients to filter inbox items 2023-12-01 12:11:09 +08:00
Hongbo Wu
bd3945fc7e
Merge pull request #3164 from omnivore-app/feature/fetch-content-for-following 
feat: fetch content for rss feed items in following folder
 2023-11-28 12:07:17 +08:00
Hongbo Wu
af5040a518 saving posts from telegram channel as rss items 2023-11-24 17:23:35 +08:00
Hongbo Wu
8075cfd431 allow adding telegram channel as feed 2023-11-24 16:06:24 +08:00
Hongbo Wu
989f4f04e0 add scan feeds api to scan feeds from opml or html 2023-11-24 12:11:33 +08:00
Hongbo Wu
022d0f9e66
Merge pull request #3159 from omnivore-app/fix/nested-labels 
feat: wildcard search for labels and nested labels
 2023-11-23 21:44:44 +08:00
Hongbo Wu
fd781644f1 feat: fetch content for rss feed items in following folder 2023-11-23 18:03:25 +08:00
Hongbo Wu
750b59cb8d
Merge pull request #3139 from omnivore-app/change-folder-to-state-1 2023-11-23 12:31:37 +08:00
Hongbo Wu
46f3515698 feat: wildcard search for labels and nested labels 2023-11-22 23:46:55 +08:00
Hongbo Wu
4e60396d17 save thumbnail of the following item 2023-11-20 17:28:34 +08:00
Hongbo Wu
e26ed0b6f0 add RSS label to the following feed items 2023-11-17 18:26:00 +08:00
Hongbo Wu
c5a97c7a41 change archive and trash from folder to state 2023-11-17 13:51:33 +08:00
Hongbo Wu
d5d3c72196
Merge pull request #3135 from omnivore-app/change-folder-to-state 
Change category to folder in saved search
 2023-11-17 12:31:21 +08:00
Hongbo Wu
fca406bb3d allow updating taskName 2023-11-17 11:27:30 +08:00
Hongbo Wu
24f31ce1eb add folder to the updates api params 2023-11-17 11:27:30 +08:00
Hongbo Wu
0869cfd477 replace category with folder in the GQL type 2023-11-16 17:50:30 +08:00
Hongbo Wu
d82eda1f32 allow importing all or unarchived only pocket items 2023-11-16 16:37:14 +08:00
Hongbo Wu
25d0123007 fix tests 2023-11-15 17:41:28 +08:00
Hongbo Wu
f9b389edad add folder to article and search item type 2023-11-15 17:25:14 +08:00
Hongbo Wu
12c18a3661 add preview content to the search API response 2023-11-15 15:53:01 +08:00
Hongbo Wu
8b92e96db1 generate slug 2023-11-15 14:27:54 +08:00
Hongbo Wu
c6807a554a item_id and saved_at are optional in the post request 2023-11-15 12:34:35 +08:00
Hongbo Wu
c59b4ca97e
Merge pull request #3054 from omnivore-app/feature/social-feeds 
feature: following
 2023-11-15 11:19:14 +08:00
Hongbo Wu
609d30adbb compatiblity with old saved searches 2023-11-15 10:22:58 +08:00
Hongbo Wu
3d84108202 remove feeds table 2023-11-14 15:27:58 +08:00
Hongbo Wu
2ade0698c5 remove unused code 2023-11-10 23:03:01 +08:00
Hongbo Wu
59bbc5562b fix tests 2023-11-10 22:37:14 +08:00
Hongbo Wu
adf13131db add folder to library_item table 2023-11-10 17:34:46 +08:00
Hongbo Wu
69dc1ba47a fix tests 2023-11-09 22:39:51 +08:00
Hongbo Wu
98cc0e2b82 upsert feed table when subscribing 2023-11-09 22:06:30 +08:00
Hongbo Wu
490b40e467 fix tests 2023-11-09 19:08:52 +08:00
Hongbo Wu
9dfc5f856d rename variables 2023-11-09 19:00:57 +08:00
Hongbo Wu
ffac5c195d update comment 2023-11-09 16:13:31 +08:00
Hongbo Wu
f96ba17c92 fetch rss content if required 2023-11-09 16:09:23 +08:00
Hongbo Wu
b7496db56c add following handler to save following item 2023-11-09 15:53:44 +08:00
Hongbo Wu
cbb5a0a921 revert searchItem changes 2023-11-08 18:16:48 +08:00
Hongbo Wu
cc53992b9a add addFollowingToLibrary API 2023-11-08 18:16:48 +08:00
Hongbo Wu
99616501cf add saveFollowing API 2023-11-08 18:16:48 +08:00
Hongbo Wu
3ff9ce3a09 add in:following search filter 2023-11-08 18:16:48 +08:00
Hongbo Wu
98faefa644 add date variables to date search 2023-11-08 18:16:48 +08:00
Hongbo Wu
2c9ec454f5 fix tests 2023-11-08 18:16:48 +08:00
Hongbo Wu
9b0b378c39 check is_fetching_content and only fetch content if required for rss feed items 2023-11-08 18:16:48 +08:00
Hongbo Wu
679512c9d1 add columns to library_item 2023-11-08 18:16:48 +08:00
Hongbo Wu
1da8dc6545 update library_item table 2023-11-08 18:16:48 +08:00
Hongbo Wu
da0d1705b5 add following graphql api to fetch user following 2023-11-08 18:16:48 +08:00
Hongbo Wu
fc783f01ec add feeds graphql api for searching feed by title or url 2023-11-08 18:16:48 +08:00
Hongbo Wu
878d881e05 add feed entity 2023-11-08 18:16:48 +08:00
Hongbo Wu
e282eea99f add feeds graphql api to the schema 2023-11-08 18:16:48 +08:00
Jackson Harper
1e336dd61c Use CLIENT_URL instead of hard coded values 2023-11-08 18:16:32 +08:00
Hongbo Wu
3e9a049306 add fields to the response of user personalization api 2023-11-07 15:39:56 +08:00
Hongbo Wu
fdad3a28af
Merge pull request #3024 from omnivore-app/integratoin 
improvement on integrations
 2023-11-03 17:20:17 +08:00
Hongbo Wu
3761b396f4 remove unused code 2023-11-03 10:17:36 +08:00
Hongbo Wu
9ef608b4b6 fix: highlights not added to the content if searching for highlighted content 2023-11-03 10:11:14 +08:00
Hongbo Wu
ffad064d3a
Merge pull request #3051 from naveennamani/patch-1 
Fixes #3050 - URLs sensitive to trailing slash
 2023-11-01 15:48:04 +08:00
Hongbo Wu
e98d029250 remove unused fields in library_item entity so we exclude them from the webhook event 2023-11-01 14:56:36 +08:00
Hongbo Wu
c93fc14de8 remove user from highlight webhook event 2023-11-01 13:26:36 +08:00
Hongbo Wu
30cc952f02 do not publish events if the highlights are recommended by club users 2023-11-01 13:24:06 +08:00
Naveen Namani
ef1adfee98
Fixes #3050 - URLs sensitive to trailing slash 
By passing false to `removeTrailingSlash` for normalize-url call, the URLs sensitive to trailing slash will be parsed correctly
 2023-10-30 14:31:46 +05:30
Hongbo Wu
8df6fb3ae2 import unarchived items by default 2023-10-30 16:22:09 +08:00
Hongbo Wu
f6c0b2ba34 add import_item_state to set integration api payload 2023-10-30 16:09:10 +08:00
Hongbo Wu
a528692125 add import_item_state to integration entity 2023-10-30 15:12:18 +08:00
Hongbo Wu
07eb97e7cc remove unused code 2023-10-30 14:41:53 +08:00
Hongbo Wu
58d5eb6bc0 create an API to create a cloud task for each active integration when the cronjob triggers 2023-10-30 14:41:52 +08:00
Hongbo Wu
5edba30e23 create an integration handler for syncing with readwise 2023-10-30 14:41:52 +08:00
Hongbo Wu
47f67c237d change source to pocket if item is imported from pocket 2023-10-30 14:41:52 +08:00
Hongbo Wu
6bdc7f01e1
Merge pull request #3027 from omnivore-app/fix/site-search 
fix: site scoped search not working for domain and hostname
 2023-10-30 11:08:26 +08:00
Jackson Harper
6db23ba05c Add intercom identity verification 2023-10-27 16:13:03 +08:00
Hongbo Wu
d480d53151 fix: site scoped search not working for domain and hostname 
* alter site_name_tsv to include hostname and domain
* add test cases
 2023-10-26 22:10:35 +08:00
Hongbo Wu
a6e8362401 add debugging logs 2023-10-26 12:56:46 +08:00
Hongbo Wu
cf95b6fa07 rename the endpoint to /prune 2023-10-25 14:41:49 +08:00
Hongbo Wu
dddb51f7d5 add test cases 2023-10-25 13:14:49 +08:00
Hongbo Wu
5f5423ddcb add a pubsub service endpoint for cleanup soft deleted users 2023-10-25 13:02:59 +08:00
Hongbo Wu
c2092e0e5d
Merge pull request #2995 from omnivore-app/fix/item-id-update 
fix: library item id could be updated if a different client request id supplied in save page api payload
 2023-10-24 15:24:18 +08:00
Jackson Harper
533e3554fd
Merge pull request #2997 from omnivore-app/feature/reading-progress-api 
ignore position check if force = true in reading progress api payload
 2023-10-24 14:45:47 +08:00
Hongbo Wu
056bf9ebf1 fix update sql 2023-10-24 13:12:59 +08:00
Hongbo Wu
3c75a23e75 fix: showing original url instead of signed url for pdf files in search api response 2023-10-24 12:02:58 +08:00
Hongbo Wu
41866d5cb7 cont 2023-10-23 22:02:43 +08:00
Hongbo Wu
3f14ac4f9b skip publishing event for importing 2023-10-23 21:31:38 +08:00
Hongbo Wu
79a647deb4 do not publish content and original html in the queue 2023-10-23 21:27:16 +08:00
Hongbo Wu
f46bb616fa update read_at to now 2023-10-23 21:14:23 +08:00
Hongbo Wu
e673db423b limit max content size to 10MB 2023-10-23 21:11:37 +08:00
Hongbo Wu
f18ea7a7ce ignore position check if force = true in reading progress api payload 2023-10-23 20:57:20 +08:00
Hongbo Wu
0c60706503 fix: date filter in bulk action api 2023-10-23 19:34:41 +08:00
Hongbo Wu
27b352f9ae fix: updates since api returns error for android client if invalid date is supplied 2023-10-23 19:07:47 +08:00
Hongbo Wu
e94a61a9bc fix: library item id could be updated if a different client request id supplied in save page api payload 2023-10-23 17:44:38 +08:00
Hongbo Wu
b26e15c5d6 lint 2023-10-23 15:44:47 +08:00
Hongbo Wu
bbc46c6389 fix: when source is rss-feed or csv-import, do not parse items in the backend 2023-10-23 15:43:19 +08:00
Hongbo Wu
850569d7e4 add test cases 2023-10-23 12:24:19 +08:00
Hongbo Wu
7763a8a939 perf: limit search item results to 100 a time and fetch labels and recommendations only needed 2023-10-23 10:52:03 +08:00
Jackson Harper
1ebfb94309
Merge pull request #2977 from Rudra-Ji/main 
fix typo
 2023-10-20 18:00:43 +08:00
Hongbo Wu
b2a61b2d7c
Merge pull request #2969 from omnivore-app/fix/query-group 
Group RSS subscriptions by feed URL and filter by scheduled time
 2023-10-20 14:45:44 +08:00
Hongbo Wu
6618a6ed19 migrate popular reads into database 2023-10-20 14:31:16 +08:00
Rudra
23eb7ea76f fix typo 2023-10-20 11:51:02 +05:30
Jackson Harper
a23b67b70b Dont rate limit auth in local env so tests can run 2023-10-20 11:44:52 +08:00
Jackson Harper
b9e2f9ee53 Add a longer rate limit window on createaccount/reset password, reduce api rate limit hits 2023-10-20 11:28:16 +08:00
Hongbo Wu
4e36a5809e fix failure of re-subscribing rss feed 2023-10-20 11:05:58 +08:00
Hongbo Wu
750ccb0c42 fix conflicts with main 2023-10-19 17:44:53 +08:00
Hongbo Wu
1880f2ace4 Update rss handler to fetch a group of rss feeds in one task 2023-10-19 17:27:55 +08:00
Hongbo Wu
f50d1c9cb9 Group RSS subscriptions by feed URL and create cloud task for the groups 2023-10-19 17:27:08 +08:00
Hongbo Wu
5283a84908 add scheduledAt to the task payload 2023-10-19 17:27:08 +08:00
Hongbo Wu
18aece84e7 add scheduledAt to subscriptions 2023-10-19 17:27:08 +08:00
Jackson Harper
c93e61ec24 Remove debug 2023-10-18 21:30:23 +08:00
Jackson Harper
78a642db6d Update type signature 2023-10-18 19:53:26 +08:00
Jackson Harper
5c576347a8 MOre debug 2023-10-18 19:26:45 +08:00
Jackson Harper
fbaaaeca7c Add some debugging 2023-10-18 18:48:18 +08:00
Jackson Harper
7848413645 Update schema 2023-10-18 16:34:20 +08:00
Jackson Harper
cf101c6d18 Cache and check feed checksums to reduce fetching 2023-10-18 16:26:03 +08:00
Hongbo Wu
46313b14ba set trust proxy = true if set in env var 2023-10-18 13:06:33 +08:00
Hongbo Wu
988b20e30c test trust proxy=true 2023-10-18 12:30:27 +08:00
Hongbo Wu
506004d410 test trust proxy=2 2023-10-18 11:58:15 +08:00
Hongbo Wu
32ba744d20 test trust proxy=1 2023-10-18 11:04:32 +08:00
Hongbo Wu
91076014cd log req.ip 2023-10-18 11:04:32 +08:00
Hongbo Wu
0874f6ff21 log x-forwarded-for header for debugging purpose 2023-10-18 11:04:32 +08:00
Jackson Harper
71aaef3628 Update attachment URL 2023-10-18 10:43:53 +08:00
Jackson Harper
b88c64424f Special handling of file URLs 2023-10-18 10:37:21 +08:00
Jackson Harper
78843fff14 Remove import 2023-10-18 10:27:09 +08:00
Jackson Harper
4220922ff8 Preserve original URL if possible 2023-10-18 10:23:26 +08:00
Jackson Harper
b6ce4f39b1 Fix imports 2023-10-18 10:15:49 +08:00
Jackson Harper
25c0051fd4 Only use signed URLs for PDF attachments 2023-10-18 10:14:35 +08:00
Hongbo Wu
fa2fba1a2c
Merge pull request #2942 from omnivore-app/fix/rules 
improvement on subscriptions and delete user api
 2023-10-17 17:14:49 +08:00
Hongbo Wu
700fd73dbb disallow social login if account is deleted 2023-10-17 17:07:46 +08:00
Hongbo Wu
6e79f6c67f update sql to deduplicate rss subscriptions 2023-10-17 16:41:14 +08:00
Hongbo Wu
eb667e04b9 update reading progress in db 2023-10-17 15:09:16 +08:00
Hongbo Wu
1b2d93e118 remove duplicate rss subscriptions by user and url; create a unique constraint 2023-10-16 17:35:29 +08:00
Jackson Harper
9e8ccb35e3 Limit users to a single import a day 2023-10-16 15:48:03 +08:00
Hongbo Wu
e54c1c81a1 soft delete user by calling delete account api 2023-10-16 15:24:53 +08:00
Jackson Harper
34798f39f0 Disable JSONLD fetching which can be quite slow 2023-10-16 11:25:37 +08:00
Jackson Harper
a20b956e26
Merge pull request #2926 from omnivore-app/fix/api-site-name-query 
Add ILIKE queries to all field queries
 2023-10-13 22:02:55 +08:00
Hongbo Wu
497b6e9f72 fetch highlights if necessary and before converting content to markdown 2023-10-13 18:27:57 +08:00
Jackson Harper
3f4be32df8 Merge library item imports 2023-10-13 18:26:02 +08:00
Hongbo Wu
a6c5f4e862 fix bulk action slow query 2023-10-13 13:45:09 +08:00
Hongbo Wu
4d64231abf save content reader in the item 2023-10-12 12:59:36 +08:00
Hongbo Wu
d1472c6519
Merge pull request #2915 from omnivore-app/fix/drop-label-position-trigger 
drop position trigger on labels table and sort labels by name returned by labels api
 2023-10-12 12:10:33 +08:00
Hongbo Wu
9b25a64fd8
Merge pull request #2916 from omnivore-app/fix/email-attachment 
fix email attachment not being saved as item
 2023-10-12 12:09:01 +08:00
Hongbo Wu
039632556b add more method in mock cloud storage and update test cases for email-attachment rest api 2023-10-12 11:56:54 +08:00
Hongbo Wu
1bc9912271 fix failure of saving highlight when position is null 2023-10-12 11:27:46 +08:00
Hongbo Wu
ebb45f1973 fix email attachment not being saved as item 2023-10-12 10:47:20 +08:00
Hongbo Wu
f1c4bc6a74 fix email attachment not being saved as item 2023-10-12 10:46:12 +08:00
Hongbo Wu
b0f0dba53d drop position trigger on labels table and sort labels by name returned by labels api 2023-10-12 10:39:35 +08:00
Hongbo Wu
30ba22cbbc add user_id to the sql query to reduce query latency 2023-10-11 16:05:35 +08:00
Hongbo Wu
a4aeb40276 fix: after successfully deleted highlights, still return error codes 2023-10-11 14:48:44 +08:00
Hongbo Wu
768845e101 update the filter value to use is:reading of the Continue Reading filter 2023-10-10 15:33:59 +08:00
Hongbo Wu
62ffaa924b add wordsCount filter 2023-10-10 12:52:49 +08:00
Hongbo Wu
93c410a564 fix is:read filter and add is:reading filter 2023-10-10 12:21:32 +08:00
Hongbo Wu
3e91f3836b fix tests 2023-10-10 10:58:48 +08:00
Hongbo Wu
835b61064f allow filter by wordsCount 2023-10-10 10:58:48 +08:00
Hongbo Wu
1adbed9876 revert postgres connection timeout 2023-10-10 10:33:49 +08:00
Hongbo Wu
bab343c41f testing 2023-10-10 10:10:45 +08:00
Hongbo Wu
ac0d295cf3 debug 2023-10-09 22:06:35 +08:00
Hongbo Wu
0b7da62aca debug 2023-10-09 21:34:52 +08:00
Hongbo Wu
f4f38e4b31 reduce unnecessary logs 2023-10-09 17:26:03 +08:00
Hongbo Wu
78490e15f1 disable db connection timeout 2023-10-09 15:30:30 +08:00
Hongbo Wu
50901d25a3 remove elastic url from env var 2023-10-09 13:00:30 +08:00
Hongbo Wu
e424328f20 fix recommendations not fetched in search results 2023-10-05 23:00:29 +08:00
Hongbo Wu
ac576aa5c7 fix isArchived = false when archivedAt is not null 2023-10-05 17:53:57 +08:00
Hongbo Wu
39b3a1cad4 resolve conflicts 2023-10-05 14:37:21 +08:00
Hongbo Wu
463231b26b fix trigger 2023-10-05 14:32:05 +08:00
Hongbo Wu
0e7f45a758 create a trigger to update recommender_names in library_item table 2023-10-05 14:32:05 +08:00
Hongbo Wu
f6c906201b fetch recommendations separately too 2023-10-05 14:32:05 +08:00
Hongbo Wu
d4404b0248 fetch labels and highlights after items fetched in a search query 2023-10-05 14:32:05 +08:00
Hongbo Wu
bb59ed757c fix word count not return 2023-10-05 14:32:05 +08:00
Hongbo Wu
cbf58614c4 use log levels in custom typeorm logger 2023-10-05 14:32:05 +08:00
Hongbo Wu
efdbc4f345 use interpolated string or dictionary parameter in logs 2023-10-05 14:32:03 +08:00
Hongbo Wu
327f6959ef fix highlight createdByMe is wrong 2023-10-05 14:31:07 +08:00
Hongbo Wu
0ff9fdef04 add elastic scroll time in the env var 2023-10-05 14:31:07 +08:00
Hongbo Wu
0912e7cf0f remove warn logs 2023-10-05 14:31:07 +08:00
Hongbo Wu
3b8b48bc21 fix recovered deleted items still shown in trash 2023-10-05 14:31:07 +08:00
Hongbo Wu
7bf3f1bbf4 set deletedAt to now when bulk delete 2023-10-05 14:31:07 +08:00
Hongbo Wu
47510c0785 fix recommendations not fetched in search results 2023-10-05 14:31:07 +08:00
Hongbo Wu
9c723ecbd3 fix id not set in page update pubsub message 2023-10-05 14:31:07 +08:00
Hongbo Wu
0c691257eb fix rules permission 2023-10-05 14:31:07 +08:00
Hongbo Wu
5f6587537e fix recommendation wildcard search 2023-10-05 14:31:07 +08:00
Hongbo Wu
bf2c54084b fix getArticle API failed on iOS 2023-10-05 14:31:07 +08:00
Hongbo Wu
9e5196406e revert rls on old tables 2023-10-05 14:31:07 +08:00
Hongbo Wu
2e5529fd81 allow readingProgressAnchorIndex to be null 2023-10-05 14:31:07 +08:00
Hongbo Wu
8712107a2f remove db caching 2023-10-05 14:31:07 +08:00
Hongbo Wu
c57ead3583 add test for when rss:feed in:archive is in the query 2023-10-05 14:31:07 +08:00
Hongbo Wu
867c31a582 add test for searching in:archive label:test 2023-10-05 14:31:07 +08:00
Hongbo Wu
6cb4bb8de8 fix in:subscription search 2023-10-05 14:31:07 +08:00
Hongbo Wu
15f9017291 fix in:archive search 2023-10-05 14:31:07 +08:00
Hongbo Wu
a35e6c0147 fix sort by read time bug 2023-10-05 14:31:07 +08:00
Hongbo Wu
dc98719b85 reduce log 2023-10-05 14:31:07 +08:00
Hongbo Wu
0c5ba53c26 fix recommendations 2023-10-05 14:31:07 +08:00
Hongbo Wu
b9af543b45 fix rss feeds 2023-10-05 14:31:06 +08:00
Hongbo Wu
673ce83c33 fix searching for site 2023-10-05 14:31:06 +08:00
Hongbo Wu
4ff5484d8e change item_type to text 2023-10-05 14:31:06 +08:00
Hongbo Wu
7c56973fad fix update filter api 2023-10-05 14:31:06 +08:00
Hongbo Wu
e696709a69 resolve conflicts 2023-10-05 14:31:06 +08:00
Hongbo Wu
7225d861f3 use skip and take instead of limit, offset 2023-10-05 14:31:06 +08:00
Hongbo Wu
68e381deb2 revert note changes 2023-10-05 14:31:06 +08:00
Hongbo Wu
fdd771bc06 add migration script 2023-10-05 14:31:06 +08:00
Hongbo Wu
dd292879a4 sort type ahead search result by saved at desc order 2023-10-05 14:31:06 +08:00
Hongbo Wu
2b9a6b26f6 debug test error 2023-10-05 14:30:37 +08:00
Hongbo Wu
a284884398 fix rebase conflicts 2023-10-05 14:30:37 +08:00
Hongbo Wu
5a9dc344d3 fix rebase conflicts 2023-10-05 14:30:35 +08:00
Hongbo Wu
cbaa3d6900 fix newsletters test 2023-10-05 14:30:10 +08:00
Hongbo Wu
c3caf15e44 fix auth tests 2023-10-05 14:30:10 +08:00
Hongbo Wu
024d558cfe update labels rls 2023-10-05 14:30:10 +08:00
Hongbo Wu
6672ca0109 fix labels tests 2023-10-05 14:30:10 +08:00
Hongbo Wu
4b95b732cf fix highlight tests 2023-10-05 14:30:10 +08:00
Hongbo Wu
b036149fe2 fix feature tests 2023-10-05 14:30:10 +08:00
Hongbo Wu
d50174ef76 fix article tests 2023-10-05 14:30:10 +08:00
Hongbo Wu
669cddeec8 fix some rls issue 2023-10-05 14:30:10 +08:00
Hongbo Wu
cea5f790d8 fix some tests 2023-10-05 14:30:09 +08:00
Hongbo Wu
01c74cb7c4 fix tests for popular reads 2023-10-05 14:29:25 +08:00
Hongbo Wu
50bfc1fa74 fix tests 2023-10-05 14:29:23 +08:00
Hongbo Wu
bab957256d fix multiple parameters in query builder 2023-10-05 14:28:52 +08:00
Hongbo Wu
e1a7a24323 improve search result 2023-10-05 14:28:52 +08:00
Hongbo Wu
1bade22076 fix file uploading 2023-10-05 14:28:52 +08:00
Hongbo Wu
e05d341952 fix file uploading 2023-10-05 14:28:52 +08:00
Hongbo Wu
86cff913fb fix label search 2023-10-05 14:28:52 +08:00
Hongbo Wu
0747c4603c fix bulk action 2023-10-05 14:28:52 +08:00
Hongbo Wu
afa1165a29 fix setting labels 2023-10-05 14:28:52 +08:00
Hongbo Wu
33ba17707b fix setting labels 2023-10-05 14:28:52 +08:00
Hongbo Wu
0016b0a9a7 fix update library item 2023-10-05 14:28:52 +08:00
Hongbo Wu
d9d71aa443 fix highlights api 2023-10-05 14:28:52 +08:00
Hongbo Wu
5ed7e819e8 fix label constraint 2023-10-05 14:28:52 +08:00
Hongbo Wu
b72acf8e6f fix permission issue 2023-10-05 14:28:52 +08:00
Hongbo Wu
4a55e801b4 fix typo 2023-10-05 14:28:50 +08:00
Hongbo Wu
3f572ea89a fix typo 2023-10-05 14:27:52 +08:00
Hongbo Wu
65bb4aa85f remove knex 2023-10-05 14:27:18 +08:00
Hongbo Wu
c1ad9b6f41 cont 2023-10-05 14:27:18 +08:00
Hongbo Wu
f8f3ec0c56 save claim in httpContext 2023-10-05 14:27:17 +08:00
Hongbo Wu
b1899e340d replace library item 2023-10-05 14:25:10 +08:00
Hongbo Wu
c655f78f8c replace highlight 2023-10-05 14:24:28 +08:00
Hongbo Wu
381e9bc173 replace database subscriber with db trigger 2023-10-05 14:24:05 +08:00
Hongbo Wu
612151c151 remove elastic from docker-compose 2023-10-05 14:24:04 +08:00
Hongbo Wu
5dce78eedb update entity subscriber 2023-10-05 14:22:58 +08:00
Hongbo Wu
648a16615b update search query 2023-10-05 14:22:58 +08:00
Hongbo Wu
751df68175 implement search function 2023-10-05 14:22:58 +08:00
Hongbo Wu
0b309b7a45 add recommendation db table and entity 2023-10-05 14:22:58 +08:00
Hongbo Wu
5f0e1af285 update highlight entity 2023-10-05 14:22:58 +08:00
Hongbo Wu
06000264d2 replace saveEmail 2023-10-05 14:22:58 +08:00
Hongbo Wu
fc52b522f2 fix test 2023-10-05 14:22:58 +08:00
Hongbo Wu
9370951893 create a repository directory 2023-10-05 14:22:56 +08:00
Hongbo Wu
425864ce01 remove knex 2023-10-05 14:21:56 +08:00
Hongbo Wu
928ed2e6a9 add libraryItemPreview entity class 2023-10-05 14:18:21 +08:00
Hongbo Wu
e1af53ab48 add libraryItem entity class 2023-10-05 14:18:21 +08:00
Hongbo Wu
789cb1d501
Merge pull request #2860 from omnivore-app/fix/self-hosting-login 
fix demo user not able to login when self hosting
 2023-10-05 12:02:46 +08:00
Hongbo Wu
c008c3154d make reminder task url optional env var 2023-10-05 11:36:34 +08:00
Hongbo Wu
16f72d6a74 fix labels and highlights were deleted when deleting items 2023-10-05 10:03:06 +08:00
Hongbo Wu
418f27f586 fix convert to email login does not change source field in user table 2023-10-04 16:35:50 +08:00
Jackson Harper
c41363b51c
Merge pull request #2831 from omnivore-app/feature/convert-to-email-api 
Update Email Feature
 2023-10-04 14:30:35 +08:00
Hongbo Wu
3cf0c45fa9 mark unsubscribed newsletter in the db 2023-10-03 12:50:55 +08:00
Hongbo Wu
ee50529f95 add update email form on account page 2023-10-02 22:21:50 +08:00
Hongbo Wu
c40e64773a convert to email api 2023-10-02 17:01:13 +08:00
Hongbo Wu
082e83b1ad fix 406 not acceptable error from subscribing an rss feed 2023-10-02 10:52:50 +08:00
Hongbo Wu
010e2841da fix getting 403 when parsing some rss url by setting a user agent in the request 2023-09-27 10:38:49 +08:00
Jackson Harper
21244a394d Move debug directory to not interfere with usernames 2023-09-22 17:56:25 +08:00
Jackson Harper
53ac43563d Set user properties instead of event properties on labels setter 2023-09-19 12:30:20 +08:00
Jackson Harper
6c6f27be3e Properly tag user on labels event 2023-09-14 16:31:01 +08:00
Jackson Harper
e4b5bd6a67 Add code to join users without email on login 2023-09-14 11:24:19 +08:00
Jackson Harper
014614d097
Merge pull request #2759 from omnivore-app/feat/web-use-posthog 
Remove segment from the web
 2023-09-13 14:01:08 +08:00
Jackson Harper
dfcfe64b1b Add a create_user event 2023-09-13 13:44:26 +08:00
Jackson Harper
0946f3d3e8 Fix some console logs, better handle posthog 2023-09-13 12:52:34 +08:00
Jackson Harper
842e4bbf98 Update help links 2023-09-13 10:22:45 +08:00
Jackson Harper
293becf596 Ignore link density checks in newsletters 2023-09-12 15:53:43 +08:00
Jackson Harper
2a3214e393 Remove identify call from backend 2023-09-12 09:53:19 +08:00
Jackson Harper
a5eb527ae1
Merge pull request #2738 from omnivore-app/fix/api-enable-rules-notifications 
Remove debounce from notification rule
 2023-09-07 18:07:06 +08:00
Jackson Harper
facde7d9b3 Remove import 2023-09-07 17:51:20 +08:00
Jackson Harper
e94529a96c
Merge pull request #2707 from Podginator/feat/saved-links 
Add Saved Searches,  the ability to add new searches, delete searches, and reorder searches.
 2023-09-07 17:27:48 +08:00
Jackson Harper
edd7246a9b Remove debounce from notification rule 2023-09-07 17:24:49 +08:00
Jackson Harper
d0583bf271 Disable persisted queries in Apollo 2023-09-07 14:49:48 +08:00
Jackson Harper
f7e5a8cbfd Increase unauthed rate limit 2023-09-07 14:01:18 +08:00
Thomas Rogers
79f42a5bde Merge Main 2023-09-06 17:54:18 +02:00
Thomas Rogers
13ab13d94d Code Review Comments 2023-09-06 17:51:42 +02:00
Jackson Harper
d2534d692b Better preview URLs for onboarding articles 2023-09-04 18:27:56 +08:00
Thomas Rogers
faf7880fcd Fix some bugs regarding scrolling, fix some UI Issues, and correct the name of Non-feed Filter. 2023-08-31 20:08:54 +02:00
Thomas Rogers
a58864fe2c Merge main 2023-08-30 19:48:04 +02:00
Thomas Rogers
d433d48365 Add Filters, and editing filters. 2023-08-30 19:37:24 +02:00
Jackson Harper
79cea907a3 Fix typo with RSS typed subscription queries 2023-08-30 14:18:39 +08:00
Jackson Harper
fe3d3be7f5 Make query more clear 2023-08-29 20:17:13 +08:00
Jackson Harper
6c1484cd09 Add type filter 2023-08-29 20:06:28 +08:00
Jackson Harper
3565c7000e Update subscriptions query 2023-08-29 17:03:49 +08:00
Jackson Harper
8851017100 Use andWhere query to not overwrite the where 2023-08-29 15:54:07 +08:00
Jackson Harper
89c17fd0ab Only filter inactive newsletters out from the subscriptions response 2023-08-29 11:34:55 +08:00
Jackson Harper
49496f7aa2 Only return active subscriptions 2023-08-29 10:19:57 +08:00
Jackson Harper
feaa539c15 Use and where for type query. Prettier changes 2023-08-28 15:23:14 +08:00
Jackson Harper
aa0f4399aa Remove prettier changes 2023-08-28 15:15:05 +08:00
Jackson Harper
38c9bbdcaa Allow getting all subscritons regardless of type at the same time 2023-08-28 15:11:16 +08:00
Hongbo Wu
ec574d0ad9 use new color or the color of the last overlap highlight when merging highlights 2023-08-18 13:54:38 +08:00
Hongbo Wu
ce480806c0 add color to create/update/merge highlight api and highlight schema 2023-08-18 13:30:54 +08:00
Hongbo Wu
cd1fc5c4db fix typo 2023-08-17 18:46:39 +08:00
Hongbo Wu
c45b760f71 fix test 2023-08-17 18:42:11 +08:00
Hongbo Wu
e54e61f3c0 reduce log size 2023-08-17 18:33:14 +08:00
Hongbo Wu
dedd76cfb7 do not publish a pubsub event if the page is imported 2023-08-17 18:24:57 +08:00
Hongbo Wu
ec88a9fe8e we don't want to create thumbnail for imported pages 2023-08-17 18:05:51 +08:00
Hongbo Wu
0609043956 reduce retry 2023-08-17 17:59:24 +08:00
Hongbo Wu
6f315ef4d9 create a transaction for uploading files in the upload_files resolver 2023-08-16 21:19:03 +08:00
Hongbo Wu
755fb66588
Merge pull request #2651 from omnivore-app/feature/csv-importer 
feature/csv importer
 2023-08-15 14:09:36 +08:00
Hongbo Wu
9ad9cea917 raise rss limit to 150 2023-08-14 22:27:43 +08:00
Hongbo Wu
5f6be169bd add savedAt and publishedAt to saveUrl api 2023-08-14 17:10:34 +08:00
Hongbo Wu
fb845152d9 add header to the pocket data csv file 2023-08-14 13:44:30 +08:00
Hongbo Wu
93cca5040e add comment 2023-08-11 14:41:21 +08:00
Hongbo Wu
05e73ebc9c reduce readwise api timeout to 5 sec 2023-08-11 14:38:45 +08:00
Hongbo Wu
b18018ab2e reduce max content size to 5MB and 10MB for both content and originalHtml 2023-08-11 14:02:29 +08:00
Hongbo Wu
792cf0b207 upgrade google cloud sdk 2023-08-11 13:55:29 +08:00
Hongbo Wu
55841e595f fix dependecies 2023-08-11 11:12:03 +08:00
Hongbo Wu
61f553fd09
Merge pull request #2616 from omnivore-app/fix/thumbnail-delay 
fix thumbnail not pre-cached if it was already set
 2023-08-10 18:45:48 +08:00
Hongbo Wu
7bd747601b update thumbnail if failed to fetch and return url if already signed 2023-08-10 18:41:10 +08:00
Hongbo Wu
f06d2558c2 update thumbnail if failed to fetch and return url if already signed 2023-08-10 18:41:03 +08:00
Hongbo Wu
8809c9764a max content length = 10MB 2023-08-10 18:12:29 +08:00
Hongbo Wu
fc76897df3 timeout = 5 when syncing omnivore data to readwise 2023-08-10 15:23:03 +08:00
Hongbo Wu
d4ac069503 fix webhook logs 2023-08-10 15:07:00 +08:00
Hongbo Wu
6acedb64c8 fix maximum call stack size error 2023-08-10 14:18:29 +08:00
Hongbo Wu
7506e27dd0 create or update an opt-in record with null grantedAt if exceeding max users 2023-08-10 13:47:39 +08:00
Hongbo Wu
163dbc73a6 do not send content in the thumbnail task payload 2023-08-10 11:55:59 +08:00
Hongbo Wu
51e0dfde23 set new limit = 1500 2023-08-10 09:43:40 +08:00
Hongbo Wu
3df0ffc5d3 fix existing opted in users not being able to be granted with the feature after increasing the limit 2023-08-08 22:08:37 +08:00
Hongbo Wu
3570079824 fix thumbnail not pre-cached if it was already set 2023-08-07 18:20:59 +08:00
Hongbo Wu
55e378c9b5
Merge pull request #2572 from omnivore-app/feature/pause-rss-feed 
add a button to pause rss feed
 2023-08-03 13:33:11 +08:00
Hongbo Wu
f4e537a4c7 set max redirect = 5 and timeout = 30 seconds 2023-08-03 13:31:18 +08:00
Hongbo Wu
82d820ea8f get link following the order of preference: via, self, alternate 2023-08-03 13:05:26 +08:00
Hongbo Wu
f3994a8a06 fix test 2023-08-02 22:00:09 +08:00
Hongbo Wu
b403dea247 add a button to pause rss feed 2023-08-01 22:07:53 +08:00
Jackson Harper
3d6d421e84
Merge pull request #2566 from omnivore-app/feat/web-improved-cards 
WIP: grid card design
 2023-08-01 20:34:39 +08:00
Jackson Harper
c0dbbc3b86 Update icons 2023-08-01 12:31:12 +08:00
Hongbo Wu
00e7e49a4b increase the max number of rss subscriptions to 50 2023-07-31 21:44:50 +08:00
Hongbo Wu
d0ad636b7f use safe stringify in logs 2023-07-28 16:52:28 +08:00
Hongbo Wu
f20e12311a check if the page is from rss feed 2023-07-28 13:53:19 +08:00
Hongbo Wu
9bc7ed92c0 we don't want to update an rss feed page if rss-feeder is tring to re-save it 2023-07-28 12:43:04 +08:00
Hongbo Wu
2e965afd69 reformat sql query in logs 2023-07-27 17:34:12 +08:00
Hongbo Wu
99a52f8d56 replace all the console logs with logger logs 2023-07-27 16:06:44 +08:00
Hongbo Wu
808f6f3d3a remove trace 2023-07-26 22:00:37 +08:00
Jackson Harper
5bf3213e49 Dont send newsletter notifications 2023-07-26 20:53:50 +08:00
Hongbo Wu
61967fdde5 fix test 2023-07-26 18:07:25 +08:00
Hongbo Wu
ced1a711eb fix test 2023-07-26 17:12:40 +08:00
Hongbo Wu
7b36156ce6 add a trace when log size too big 2023-07-26 17:07:24 +08:00
Hongbo Wu
8cd47eec11 run prettier 2023-07-26 16:41:55 +08:00
Hongbo Wu
07d43812a7 use winston logger instead of console log so the log will be truncated if it exceeds the max entry size 2023-07-26 16:27:18 +08:00
Hongbo Wu
66490df72f deeply truncate log object size 2023-07-26 15:52:13 +08:00
Hongbo Wu
f0e62b1d2a use lodash instead of standalone lodash.debounce package 2023-07-26 13:41:51 +08:00
Hongbo Wu
0a227decdd
Merge pull request #2515 from omnivore-app/fix/push-notification 
fix/push notification
 2023-07-26 12:36:13 +08:00
Hongbo Wu
2841783504 limit max log entry size to 256kb 2023-07-25 23:39:03 +08:00
Hongbo Wu
f5ae16a904 disable gql request log 2023-07-25 23:03:57 +08:00
Hongbo Wu
ecd6a2922f disable sentry error handler 2023-07-25 22:32:00 +08:00
Hongbo Wu
fd6223ee93 revert changes 2023-07-25 21:20:26 +08:00
Hongbo Wu
a3e86a9bc1 only log error.message 2023-07-25 20:52:15 +08:00
Hongbo Wu
0e38060b0c set maxEntrySize for logger to 256KB 2023-07-25 20:07:02 +08:00
Hongbo Wu
1006cdad36 reduce error log size 2023-07-25 19:11:08 +08:00
Hongbo Wu
de647e44bd Revert "move sentry error handler after apollo middleware" 
This reverts commit 8d8d2b592c.
 2023-07-25 19:10:12 +08:00
Hongbo Wu
8d8d2b592c move sentry error handler after apollo middleware 2023-07-25 18:18:40 +08:00
Hongbo Wu
77456373bf return content without highlight if it takes too long to process highlighted content 
* timeout if the get text node function takes more than 10 minutes
 2023-07-25 11:19:06 +08:00
Jackson Harper
fa1ff9ba17 Upgrade node to 18.16 2023-07-24 15:55:11 +08:00
Hongbo Wu
0d3b742837 limit 20k articles per pocket import 2023-07-21 18:03:57 +08:00
Hongbo Wu
e0a6a2e0a4 limit number of rss subscriptions to 20 2023-07-21 15:28:20 +08:00
Hongbo Wu
99b12c6cfb debounce send push notificaiton within 1 minute 2023-07-20 21:48:30 +08:00
Hongbo Wu
9a224d2324 add event_types and fix the validation of form on rules page 2023-07-20 18:33:30 +08:00
Hongbo Wu
4dad4a6fa3 revert push notification 2023-07-20 17:29:26 +08:00
Hongbo Wu
6004b51956 return not found if rss feed not found 2023-07-20 14:21:36 +08:00
Hongbo Wu
7ceb96a93e fix edit rss ui 2023-07-20 13:35:43 +08:00
Hongbo Wu
400739f61c stop writing empty csv file if no new data retrieved from the integration 2023-07-20 10:32:46 +08:00
Hongbo Wu
6ef74a2601 disable import button while importer is running 2023-07-20 09:52:23 +08:00
Hongbo Wu
cd440a77c3 send last fetched at as a unix timestamp in ms 2023-07-19 14:09:38 +08:00
Hongbo Wu
35c074e647 create a cloud task to fetch rss feed item for the new subscription 2023-07-19 10:07:08 +08:00
Hongbo Wu
dcb306bcfc do not include archived items in library or subscription 2023-07-18 16:45:24 +08:00
Hongbo Wu
456a73f4ab make library an internal label 2023-07-17 18:29:35 +08:00
Hongbo Wu
44c9bfce48
Merge pull request #2494 from omnivore-app/fix/rss-feed 
fix/rss feed
 2023-07-17 17:50:00 +08:00
Hongbo Wu
3cfd390346
Merge pull request #2489 from omnivore-app/more-search-scope 
add in:subscription and in:library in search scope
 2023-07-17 17:28:24 +08:00
Hongbo Wu
03ce4b00d2 add authorization header in rss feed fetching request 2023-07-17 17:27:12 +08:00
Hongbo Wu
be1e8169ba exclude items with newsletter or rss labels in library 2023-07-17 17:06:39 +08:00
Hongbo Wu
113720caf6 wrap document fragment in the <html> and <body> in order to parse the HTML correctly 2023-07-17 16:14:59 +08:00
Hongbo Wu
b3dcf0ee46 default return builder 2023-07-17 16:14:03 +08:00
Hongbo Wu
e3af3f526c add in:subscription and in:library in search scope 2023-07-14 17:07:25 +08:00
Hongbo Wu
153662ce6a set savedAt and publishedAt timestamp for rss item 2023-07-14 12:21:34 +08:00
Hongbo Wu
983f0dcb99 add rss to the search scope 2023-07-13 11:09:13 +08:00
Hongbo Wu
9697ae2eca make rss an internal label 2023-07-13 10:58:37 +08:00
Hongbo Wu
87275fbb9c use a separate queue for fetching rss feed item content 2023-07-13 10:27:15 +08:00
Hongbo Wu
3444480b95 change the rss-feed cronjob api endpoint 2023-07-12 10:53:32 +08:00
Hongbo Wu
435fa0c541
Merge pull request #2478 from omnivore-app/fix/puppeteer-timezone 
add optional locale and timezone to saveUrl request for puppeteer to use
 2023-07-11 17:32:10 +08:00
Hongbo Wu
1f283e6122 add optional locale and timezone to saveUrl request for puppeteer to use 2023-07-11 16:15:32 +08:00
Hongbo Wu
b1b4a2e3fd save rss feed url in the index 2023-07-11 13:14:43 +08:00
Hongbo Wu
c500997693 add lastFetchedAt to the cloud task payload 2023-07-11 13:05:27 +08:00
Hongbo Wu
eb9a3eddd0 skip old item 2023-07-11 12:59:55 +08:00
Hongbo Wu
44473ba089 upload feed subscriptions in cloud storage in cronjob 2023-07-11 12:22:49 +08:00
Hongbo Wu
56a27878dc add updateSubscription api 2023-07-10 15:24:33 +08:00
Hongbo Wu
68667053c8 create an endpoint to fetch all rss feeds 2023-07-07 21:15:42 +08:00
Hongbo Wu
574636451d update api to accommodate rss feed subscriptions 2023-07-06 19:51:30 +08:00
Hongbo Wu
7075375e2b fix test 2023-07-06 19:51:30 +08:00
Hongbo Wu
2adf753f8f update graphql schema 2023-07-06 19:51:30 +08:00
Hongbo Wu
a6192ed86b add rss subscription entity class 2023-07-06 19:51:30 +08:00
Jackson Harper
0c96172053
Merge pull request #2458 from omnivore-app/feature/search-filter-cat 
feat: add category field to Filter object
 2023-07-05 10:41:43 -07:00
Hongbo Wu
599d61016e feat: add category field to Filter object 2023-07-03 19:06:45 +08:00
Hongbo Wu
e4284d215c feat: boost sitename score by 2 times in the basic search 2023-07-03 18:09:41 +08:00
Hongbo Wu
afaaf8af60 feat: allow searching by note in the basic search 2023-07-03 18:06:54 +08:00
Hongbo Wu
a413521e3b
Merge pull request #2445 from omnivore-app/feature/create-labels 
feat: create labels if not exist when setting labels
 2023-07-03 16:54:15 +08:00
Hongbo Wu
88dc514d27 feat: allow recovering a soft deleted item through updatePage api 2023-06-30 18:39:43 +08:00
Hongbo Wu
ba35fa4447 feat: show soft deleted pages within 14 days by searching in:trash 2023-06-30 18:05:41 +08:00
Hongbo Wu
6be9afb619 feat: allow searching by note: 2023-06-30 16:02:49 +08:00
Hongbo Wu
53b854b69f fix: improve full-text search result 
* always sort by score first
* boost title score by 3 times
* tie breaker is 0.3
 2023-06-30 15:32:09 +08:00
Hongbo Wu
13afb7febd feat: create labels if not exist when setting labels in the highlight 2023-06-30 11:40:39 +08:00
Hongbo Wu
8d3cac989b feat: create labels if not exist when setting labels in the page 2023-06-30 11:08:03 +08:00
Hongbo Wu
74f2dd6bf1 catch authorization error in rate limiter and 5 rpm 2023-06-28 17:04:47 +08:00
Hongbo Wu
b7c133d58e fix: email sending and recommendation 
* use token in the custom header as the key in rate limiter
 2023-06-28 17:00:11 +08:00
Hongbo Wu
b60ace4877 log unauthorized error 2023-06-28 12:22:32 +08:00
Hongbo Wu
1b069c2696 catch authorization error and return 401 2023-06-28 12:19:47 +08:00
Hongbo Wu
af7350bd31 fix: importer status update email not sent 
* create omnivore-authorization header to contain the token
 2023-06-28 11:00:40 +08:00
Hongbo Wu
7c0c4eaa40 fix: type error 2023-06-27 21:05:00 +08:00
Hongbo Wu
ed8287df19 remove tracking params from tweet url 2023-06-27 17:49:31 +08:00
Hongbo Wu
bb359d9747 save normalized url 2023-06-27 16:58:57 +08:00
Hongbo Wu
f372636d6e fix: tweet not saved correctly when using share button on iOS 2023-06-27 16:47:36 +08:00
Hongbo Wu
90444788ba
Merge pull request #2400 from omnivore-app/feat/web-multi-select-labels 
Add support for multi select Add Label
 2023-06-22 09:42:59 +08:00
Jackson Harper
cceff32e5f Return non-error if setting labels is a noop 2023-06-21 20:24:36 +08:00
Hongbo Wu
826565a2c4 temporarily skip unsubscribing by url and mark them as unsubscribed automatically 2023-06-21 12:13:22 +08:00
Hongbo Wu
4f74b32ff6 do not subscribe a newsletter if subscription already exists and is unsubscribed 2023-06-21 12:02:15 +08:00
Hongbo Wu
84788e7551 fix: mark failed to unsubscribe newsletters as unsubscribed state 2023-06-21 11:33:49 +08:00
Hongbo Wu
ea6d06f464
Merge pull request #2375 from omnivore-app/fix/pocket-importer 
fix pocket paginated api
 2023-06-19 11:19:31 +08:00
Hongbo Wu
0d8a3f7346 fix pocket paginated api 2023-06-16 18:11:31 +08:00
Hongbo Wu
c8e3f79655
Merge pull request #2368 from omnivore-app/fix/api-robots-txt 
Add robots.txt to the list of reserved names
 2023-06-16 16:38:06 +08:00
Jackson Harper
cf7c50468e Use normalized URLs when querying article saving requests 
When we save pages we save the normalized URL, so if a user saves
a URL like https://foo.com/ in elastic it will be saved as
https://foo.com but the browser will redirect to this url:

https://omnivore.app/article?url=https://foo.com/ and the URL
lookup will fail.
 2023-06-16 15:15:24 +08:00
Jackson Harper
5dd28de9e8 Add security.txt file too 2023-06-16 12:54:30 +08:00
Jackson Harper
52af2d8cb4 Add robots.txt to the list of reserved names 2023-06-16 12:54:30 +08:00
Hongbo Wu
9a3673e315
Merge pull request #2367 from omnivore-app/fix/refresh-index 
refresh index after altering the page state or adding/removing/updating labels/highlights
 2023-06-15 12:18:53 +08:00
Hongbo Wu
ceaa7b0356 refresh index after altering the page state or adding/removing/updating labels/highlights 2023-06-15 11:37:09 +08:00
Hongbo Wu
5d2ab3f554 create another rate limiter for auth api with 4 rpm 2023-06-14 19:31:18 +08:00
Hongbo Wu
fc08a434e1
Merge pull request #2359 from omnivore-app/fix/pocket-import 
fix/pocket import
 2023-06-14 15:32:10 +08:00
Jackson Harper
aaf6bb617f Remove backend events that we dont need to watch 2023-06-14 13:19:39 +08:00
Hongbo Wu
11cf9f7c28 stringify csv before uploading 2023-06-14 12:50:06 +08:00
Hongbo Wu
d851a705a6 refresh index after archiving page 2023-06-13 12:44:32 +08:00
Hongbo Wu
771114b30d refresh index if update is not async 2023-06-13 12:43:18 +08:00
Hongbo Wu
1a48ad03e4 use a separate queue for thumbnailer 2023-06-12 12:25:04 +08:00
Hongbo Wu
8904247620 add test 2023-06-09 18:58:41 +08:00
Hongbo Wu
b12e127d15 use gold color for the label 2023-06-09 17:36:38 +08:00
Hongbo Wu
5db4354f3f make internal optional in label response 2023-06-09 17:33:14 +08:00
Hongbo Wu
bd4204fc42 add analytics 2023-06-09 13:14:14 +08:00
Hongbo Wu
55904ccf04 feat: add set favorite article api which adds the favorites internal label to it 2023-06-09 13:09:47 +08:00
Hongbo Wu
6116746e80 feat: make newsletters and favorites internal labels 2023-06-09 12:48:46 +08:00
Hongbo Wu
51032d2bf1 fix: test 2023-06-08 18:52:47 +08:00
Hongbo Wu
039f00e3b2 perf: add max doc and throttling to bulk api 2023-06-08 18:36:09 +08:00
Hongbo Wu
4678d6f473 check label uniqueness by name 2023-06-08 13:33:59 +08:00
Hongbo Wu
3630d743d7 add unique labels 2023-06-08 13:29:05 +08:00
Hongbo Wu
de3623d3f2 allow mark as read and add labels bulk actions 2023-06-08 13:05:30 +08:00
Hongbo Wu
1af845e4e5 feat: add bulk action api schema 2023-06-07 21:50:43 +08:00
Hongbo Wu
b698029972 store token in cookie instead of authorization header 2023-06-07 14:24:52 +08:00
Hongbo Wu
c0872939a7
Merge pull request #2311 from omnivore-app/feature/metadata 
Feature: Set thumbnail and pre-cache all the images in the content
 2023-06-07 12:08:20 +08:00
Hongbo Wu
9be84fb473 create thumbnail task for newsletters too 2023-06-07 11:52:13 +08:00
Hongbo Wu
986c4eb48b fix: parse unsubscribe mailto email address and set subject 2023-06-06 19:53:18 +08:00
Hongbo Wu
4f4f50af49 fix: redundant argument 2023-06-06 12:05:33 +08:00
Hongbo Wu
0c3fa2cdd4 feat: start a cloud task to set thumbnail and pre-cache all the images in the content 2023-06-06 11:56:31 +08:00
Hongbo Wu
e2315b5662 feat: add thumbnail handler 2023-06-06 11:02:46 +08:00
Hongbo Wu
c66d13177f
Merge pull request #2295 from omnivore-app/fix/twitter-handler-timeout 
Feature: importer metrics collector and fix puppeteer timeout issue
 2023-06-05 17:17:50 +08:00
Jackson Harper
e53cc28683 Fix issue where we return EPUB content reader for items marked as book in metadata 
If we parse a page with <meta property="og:type" content="book"> in
it now, we incorrectly set this to ContentReader.EPUB, which then
causes syncing issues on iOS which doesn't have an EPUB reader
type defined in its GraphQL schema yet.
 2023-06-05 14:44:51 +08:00
Hongbo Wu
a439aac7e7 feat: update api to include preview image 2023-06-05 12:38:30 +08:00
Hongbo Wu
d69e732579 fix: dedupe url if failed to save 2023-06-01 16:11:35 +08:00
Hongbo Wu
15b41560de fix: improve the speed of generating highlighted markdown content 2023-05-26 15:51:54 +08:00
Hongbo Wu
15b977f7c6 fix: set server timeout = 10 mins 2023-05-26 15:00:46 +08:00
Jackson Harper
751aec2676 Use number instead of function 2023-05-19 18:10:28 +08:00
Jackson Harper
f671b3625c Change rate limiting rules 2023-05-19 18:05:57 +08:00
Hongbo Wu
4cc9547935 fix: fallback to html if failed to convert html to markdown 2023-05-19 16:55:50 +08:00
Hongbo Wu
87f55eabf6 fix: sentry report on node is null 2023-05-19 10:54:43 +08:00
Hongbo Wu
331ddf69d4 remove debugging log 2023-05-18 21:12:44 +08:00
Hongbo Wu
0adab7ad23 fix: sentry report on unable to find highlight 2023-05-18 21:07:51 +08:00
Hongbo Wu
48554be76d fix: wrap link with == 2023-05-18 21:04:27 +08:00
Hongbo Wu
2bf932cbdc fix: &npsp; breaks highlight 2023-05-18 16:50:49 +08:00
Jackson Harper
058ccf3006
Merge pull request #2183 from omnivore-app/fix/unauthed-rate-limit 
Reduce the rate limit for unauthenticated users
 2023-05-18 15:15:18 +08:00
Jackson Harper
1d37f04808 Properly call getClaimsByToken 2023-05-16 17:14:38 +08:00
Hongbo Wu
e14080a05f Set labels = undefined if pocket tags is undefined 2023-05-16 11:04:22 +08:00
Hongbo Wu
62ce7ff713
Merge pull request #2200 from omnivore-app/fix/pocket-import 
Pass pocket auth token in query string
 2023-05-15 21:53:15 +08:00
Hongbo Wu
175b82485f Fix typo 2023-05-15 17:39:29 +08:00
Hongbo Wu
c2e62155cc Pass pocket auth token in query string 2023-05-15 16:56:27 +08:00
Hongbo Wu
e3d8fae6c5 Remove debug log 2023-05-12 19:28:19 +08:00
Hongbo Wu
791d11f581 Allow content to be converted into highlightedMarkdown format 2023-05-12 19:27:04 +08:00
Jackson Harper
c49ca0149f Set max to 10/minute for unauthenticated requests 2023-05-12 17:11:20 +08:00
Hongbo Wu
565c932a2d Add archivedAt to the search item in the search api response 2023-05-12 12:22:11 +08:00
Jackson Harper
cbdd4f0a71
Merge pull request #2184 from omnivore-app/fix/rm-auth-cookie-on-reset 
Clear auth cookie when password reset is triggered
 2023-05-11 17:33:16 +08:00
Jackson Harper
8d9f54c62c Fix linting 2023-05-11 13:36:27 +08:00
Jackson Harper
78681df4ab Use clearCookie, rm the pending auth cookie too 2023-05-11 13:32:43 +08:00
Jackson Harper
4329c28904 On password reset clear the auth cookie 2023-05-11 13:29:35 +08:00
Jackson Harper
88aa7aad4c Reduce the rate limit for unauthenticated users 2023-05-11 13:23:43 +08:00
Jackson Harper
f8df71508b Enforce max lengths on signup credentials 2023-05-10 23:09:31 +08:00
Hongbo Wu
a4d385bc83 add sorting by wordsCount 2023-05-03 08:36:33 +08:00
Hongbo Wu
31cbcc7c7f validate sign up request body in mobile sign-up router 2023-04-27 12:18:32 +08:00
Hongbo Wu
0e8887a1d9 trim user email before saving in db 2023-04-27 11:39:34 +08:00
Hongbo Wu
673c43aee0 trim user email in mobile sign-in router 2023-04-27 11:37:27 +08:00
Hongbo Wu
fb4c8cb1ed trim user email in mobile sign-up router 2023-04-27 11:33:03 +08:00
Hongbo Wu
692fd7b959 Fix save url error when uid is not in the apollo context 2023-04-25 17:01:25 +08:00
Hongbo Wu
3841734deb Fix forwarded email subject being treated as an URL, for example "Fwd: subject" 2023-04-24 21:16:44 +08:00
Hongbo Wu
b891e547c9 Make content optional 2023-04-24 21:03:39 +08:00
Hongbo Wu
e512ba0bad Fix tests 2023-04-24 18:04:18 +08:00
Hongbo Wu
69968e5288 Save url if the email subject is a parsable url 2023-04-24 12:54:23 +08:00
Hongbo Wu
bce7afde50 Eagerly load profile of the user 2023-04-24 12:53:40 +08:00
Hongbo Wu
b57ccd0616 Allow wildcards in query filter 2023-04-21 17:39:04 +08:00
Hongbo Wu
b2c115f9d9 Allow wildcards in label: filter 2023-04-21 17:10:46 +08:00
Hongbo Wu
130677b50c Add contentReader to the typeAheadSearchResult 2023-04-20 10:00:56 +08:00
Hongbo Wu
5ce4bf7171 Compare labels with lower cased 2023-04-20 09:38:04 +08:00
Hongbo Wu
62b772eeca Allow saving epub from email attachments 2023-04-19 11:30:49 +08:00
Jackson Harper
3c12162866 Fix typo in content type check 2023-04-18 11:22:44 +08:00
Jackson Harper
69df32d428 Implement a reader for epubs 2023-04-18 09:00:48 +08:00
Hongbo Wu
e6690709c3 Replace express-http-context with express-http-context2 2023-04-12 17:50:13 +08:00
Hongbo Wu
014dc773e3 Fix labels not saved correctly by saveUrl 2023-04-12 17:04:59 +08:00
Hongbo Wu
874247661b resolve conflicts 2023-04-12 17:04:59 +08:00
Hongbo Wu
c07ada0218 resolve conflicts 2023-04-12 17:04:59 +08:00
Hongbo Wu
42dc3f7330 Fix some external requests not being mocked 2023-04-12 17:04:59 +08:00
Hongbo Wu
9857af27e2 Add labels and state to createArticle API 2023-04-12 17:04:59 +08:00
Hongbo Wu
abd42f7064 Add labels and state to saveUrl API 2023-04-12 16:50:11 +08:00
Hongbo Wu
ce273b5172 Add test case for archives page and saves labels in savePage API 2023-04-12 16:50:11 +08:00
Hongbo Wu
87d3f5b3d8 Add labels and state to savePage API 2023-04-12 16:50:11 +08:00
Hongbo Wu
fcf08fc364 Add state and labels to savePageInput 2023-04-12 16:50:11 +08:00
Hongbo Wu
4b578bbebb Process labels and state in the csv file 2023-04-12 16:50:11 +08:00
Hongbo Wu
9a46e935d9 Add [] to the labels in csv 2023-04-12 16:50:11 +08:00
Hongbo Wu
06b6583f29 Mock cloud storage bucket 2023-04-12 16:50:11 +08:00
Hongbo Wu
76b63efdc1 Fix tests 2023-04-12 16:50:11 +08:00
Hongbo Wu
10c01c12f2 Add test case for integration import task handler 2023-04-12 16:50:10 +08:00
Hongbo Wu
56fd3d7617 Create a router to handle importing from integrations cloud task 2023-04-12 16:50:10 +08:00
Hongbo Wu
503ec3d413 Update task name 2023-04-12 16:50:10 +08:00
Hongbo Wu
9ff0711a73 Remove wrong comments 2023-04-12 16:50:10 +08:00
Hongbo Wu
c8ec84562b Create an import from integration cloud task 2023-04-12 16:50:10 +08:00
Hongbo Wu
b62b769604 Paginate pocket api 2023-04-12 16:50:10 +08:00
Hongbo Wu
1c5991aded import from integration api return a boolean value 2023-04-12 16:50:10 +08:00
Hongbo Wu
c78fe9f87d Paginate pocket retrieve api response data 2023-04-12 16:50:10 +08:00
Hongbo Wu
737957ff82 Add import from pocket to web 2023-04-12 16:50:10 +08:00
Hongbo Wu
2e5e3bd19e Add headers 2023-04-12 16:50:10 +08:00
Hongbo Wu
59621768c8 Get request token from pocket 2023-04-12 16:50:10 +08:00
Hongbo Wu
ae2f695df8 Get access token from pocket 2023-04-12 16:50:10 +08:00
Hongbo Wu
9c7f6fefca Add PocketIntegration service to the list 2023-04-12 16:50:10 +08:00
Hongbo Wu
c3bfb593c6 Fix regression 2023-04-12 16:50:10 +08:00
Hongbo Wu
02247bc0a6 Add import from integration api implementation 2023-04-12 16:50:10 +08:00
Hongbo Wu
e2d050288e write the list of urls to a csv file and upload it to gcs 2023-04-12 16:50:10 +08:00
Hongbo Wu
94d8903ec1 Add pocket integration support 2023-04-12 16:50:10 +08:00
Hongbo Wu
0bf4119c92 Refactor with factor pattern 2023-04-12 16:50:10 +08:00
Hongbo Wu
4ce4cd0a62 Add a type for export or import to the integration table 2023-04-12 16:50:10 +08:00
Hongbo Wu
9056318667 Change type field in integration table from enum to text type 2023-04-12 16:50:10 +08:00
Hongbo Wu
b43f6904f4
Merge pull request #2021 from omnivore-app/fix/subscriptions 
creates or updates subscription only if their is a valid unsubscribe link
 2023-04-12 13:17:34 +08:00
Hongbo Wu
80d0e7c944 Move pageType to function_resolver 2023-04-12 12:37:22 +08:00
Hongbo Wu
158a75807c Use camel case 2023-04-12 12:27:05 +08:00
Hongbo Wu
4969391d4c Replace picture with image 2023-04-12 12:15:53 +08:00
Hongbo Wu
c5eb223c07 Add tweet, video, picture pageType 2023-04-12 11:53:23 +08:00
Hongbo Wu
f28912728a creates or updates subscription only if their is a valid unsubscribe link 2023-04-11 22:39:47 +08:00
Hongbo Wu
83cd0ac880 Return error if query exceeds 255 char 2023-04-06 12:03:26 +08:00
Hongbo Wu
3d599ad7ab Remove sanitize directive on query argument because we will change the argument type if doing so 2023-04-06 11:56:19 +08:00
Hongbo Wu
c596e88f46 Set search query max size to 255 char 2023-04-05 12:20:52 +08:00
Hongbo Wu
e5ffd3f5bd Stop inserting search history to the DB in the search API 2023-04-05 11:45:39 +08:00
Hongbo Wu
819d275497 Add wordsCount if undefined when creating pages in elastic 2023-04-04 12:52:46 +08:00
Hongbo Wu
32d31af847
Merge pull request #1970 from omnivore-app/double-quote-size-limit 
Double max length of quote in highlights
 2023-04-04 11:44:28 +08:00
Hongbo Wu
c52d2f0f56 Fix pageType to be null in updatesSince api response 2023-04-04 11:00:13 +08:00
Hongbo Wu
8d79f858c3 Double max length of quote in highlights 2023-04-03 18:46:59 +08:00
Hongbo Wu
53a2df5dd2 Fix start_date or end_date is undefined in elasticsearch search query 2023-03-27 18:45:24 +08:00
Hongbo Wu
dd24cfbeec Remove async 2023-03-23 10:45:39 +08:00
Hongbo Wu
abb8df26e5 Set HIGHLIGHT as default highlight type in every highlight 2023-03-23 10:44:30 +08:00
Hongbo Wu
aae16d7a4d Set HIGHLIGHT as default highlight type 2023-03-22 22:02:03 +08:00
Hongbo Wu
b3a965781c resolve conflicts 2023-03-22 10:40:03 +08:00
Hongbo Wu
ee151637e3 Only send event if highlight updated 2023-03-22 10:37:07 +08:00
Hongbo Wu
9df1546323 Fix tests 2023-03-22 10:37:07 +08:00
Hongbo Wu
b0e91cbbbb Change GraphQL APIs 2023-03-22 10:36:38 +08:00
Hongbo Wu
71826148fd Add type to the highlight in the elasticsearch and three types are HIGHLIGHTS, REDACTION and NOTE 2023-03-22 10:34:40 +08:00
Hongbo Wu
013b460e39 Check the permission to access page in elasticsearch 2023-03-21 15:56:30 +08:00
Hongbo Wu
e84b517493 Do not update createdAt for existing page 2023-03-21 13:57:57 +08:00
Hongbo Wu
8148453503 When saving an existing page, update the content 2023-03-21 13:44:00 +08:00
Hongbo Wu
bc9ba1d976 Reset state to processing when creating articles 2023-03-21 13:05:00 +08:00
Hongbo Wu
5af9d9266c Do not fetch deleted article in saving request 2023-03-21 13:03:53 +08:00
Hongbo Wu
f053b60cbe Allow array values in page filters 2023-03-21 13:03:23 +08:00
Hongbo Wu
f93148fcdd Add BAD_DATA to the saving request error code 2023-03-21 13:02:38 +08:00
Jackson Harper
265c050f5c Return readerURL on save URL requests 2023-03-20 17:28:55 +08:00
Hongbo Wu
c315e8be04 Return article's url in saveUrl api 2023-03-20 16:49:18 +08:00
Hongbo Wu
fb945f2791 Return url in save API 2023-03-20 16:48:50 +08:00
Hongbo Wu
f348ec7100 Add id to saving request and make both id and url optional 2023-03-17 17:56:38 +08:00
Hongbo Wu
f9396450a5 Get saving request by url 2023-03-17 11:37:59 +08:00
Hongbo Wu
fc662dd2d5 Return existing slug if page exists 2023-03-17 11:37:59 +08:00
Hongbo Wu
86ae533b66
Merge pull request #1915 from omnivore-app/allow-scoping-searches-to-site 
Allow scoping searches to site:
 2023-03-17 09:59:21 +08:00
Hongbo Wu
adae20e7e1 Use elasticsearch body builder 2023-03-15 19:15:40 +08:00
Hongbo Wu
45eed5c40d Must match siteName 2023-03-15 13:32:04 +08:00
Hongbo Wu
7d2cb07af0 eslint 2023-03-14 22:24:17 +08:00
Hongbo Wu
e5f1f25a4d Allow scoping searches to site: 2023-03-14 18:26:45 +08:00
Hongbo Wu
a8682b8cf6 Save html along with highlight and return 2023-03-14 15:55:32 +08:00
Hongbo Wu
bf6ae38fe1 Add html to the highlight document in elasticsearch 2023-03-14 15:20:09 +08:00
Hongbo Wu
c3f2262b6b Update test 2023-03-14 14:52:42 +08:00
Hongbo Wu
ee2487ff01 Case insensitive 2023-03-14 14:42:56 +08:00
Hongbo Wu
461b348dc0 Use lower function in the query 2023-03-14 14:42:00 +08:00
Hongbo Wu
7dd32c3e87 Trim whitespace text from user email address 2023-03-14 12:57:23 +08:00
Hongbo Wu
a0ec7bc89d Reset readingProgressAnchorIndex to zero if the input is zero 2023-03-11 10:16:47 +08:00
Hongbo Wu
98d5f92ad4 Call youtube data api to fetch published date 2023-03-10 17:38:03 +08:00
Hongbo Wu
a0aab94e8b Add site_name and published date to saved twitter 2023-03-10 16:42:40 +08:00
Hongbo Wu
5c3dd904a5 do not use highlightPositionAnchorIndex as the location 2023-03-10 15:54:51 +08:00
Hongbo Wu
12160cc9dc filter out highlights with no quote 2023-03-10 15:43:06 +08:00
Hongbo Wu
45a857152b Add minLength = 1 for quote in highlight 2023-03-10 15:40:31 +08:00
Hongbo Wu
2e197644d4 Add debug logs 2023-03-10 15:17:04 +08:00
Hongbo Wu
45b2fa6329 fix not setting zero 2023-03-10 12:46:41 +08:00
Hongbo Wu
bba0f69f62 Add top percentage to the reading position in API 2023-03-10 12:38:25 +08:00
Hongbo Wu
590a1cb8ad Only publish labels to add to the pubsub queue 2023-03-09 21:46:33 +08:00
Hongbo Wu
104da191d2 Allow webhooks with same event types 2023-03-09 15:00:06 +08:00
Jackson Harper
58156b72ff Update yarn.lock 2023-03-09 10:44:42 +08:00
Jackson Harper
d0e4c01b45 Allow passing mode to backend to better handle mode changes in client 2023-03-09 10:15:14 +08:00
Hongbo Wu
a1e3ac8a07 Add quote to the updateHighlightInput of the updateHighlight API 2023-03-08 15:37:24 +08:00
Hongbo Wu
87fd49540a
Merge pull request #1697 from omnivore-app/dependabot/npm_and_yarn/knex-2.4.2 
Bump knex from 0.21.12 to 2.4.2
 2023-03-06 09:45:50 +08:00
Hongbo Wu
0a350bef6d Fix issues with knex version bump 2023-03-05 21:36:46 +08:00
Hongbo Wu
bd12ecd7ec Add publishedAt to the updatePageInput 2023-03-02 13:40:59 +08:00
Hongbo Wu
65c244edd7
Merge pull request #1819 from omnivore-app/feature/search-filter-with-no-as-keyword 2023-02-21 12:23:22 +08:00
Hongbo Wu
214766dda6 Add no: keyword to the search filter to filter items without label or highlight 2023-02-17 12:48:02 +08:00
Hongbo Wu
153ca6995b Refactor exclude labels query 2023-02-17 12:13:20 +08:00
Hongbo Wu
7f761049d7 Preparse the newsletter content 2023-02-17 09:49:19 +08:00
Hongbo Wu
cc8b1cefdb Preserve <pre> elements with prism- class and identity them as code blocks 2023-02-14 12:33:59 +08:00
Hongbo Wu
d816ee9563 Allow image and media to be loaded in puppeteer 2023-02-13 16:50:39 +08:00
Hongbo Wu
0359cefb4f
Merge pull request #1794 from omnivore-app/add-domdistiller-test-page 
Add dom-distiller generated test pages
 2023-02-10 10:42:59 +08:00
Hongbo Wu
c285a017c3 Increase view.html iframe height 2023-02-09 22:20:58 +08:00
Hongbo Wu
761f2ff8f2 Add JWT token to the request 2023-02-09 18:05:04 +08:00
Hongbo Wu
a91cf9ef63 Add an option for the original html to be parsed by distiller 2023-02-09 17:44:02 +08:00
Jackson Harper
b0ccfc213d Operate on pages in processing state during bulk operations 2023-02-09 12:28:06 +08:00
Jackson Harper
e6833fddff Add to address on email not forwarded logs 2023-02-06 11:13:28 +08:00
Jackson Harper
1e63899748 Add additional logging for mobile auth login errors 2023-02-06 11:11:28 +08:00
Hongbo Wu
1458395630
Merge pull request #1759 from omnivore-app/archive-all-api 
bulk action api
 2023-02-02 22:14:33 +08:00
Hongbo Wu
bb5c104a42 Change archiveAll to bulkAction which allows deletion too 2023-02-02 21:26:37 +08:00
Hongbo Wu
7255061e36 Unescape HTML entities in annotation when creating highlight 2023-02-02 11:32:24 +08:00
Hongbo Wu
3b888bbb08 Make Async call to archive all 2023-02-01 18:50:10 +08:00
Hongbo Wu
03ab96b7f8 Add archiveAll API 2023-02-01 18:05:16 +08:00
Hongbo Wu
dd6c8cf6dc Make subject optional too 2023-02-01 13:53:27 +08:00
Hongbo Wu
ac6f5b497c Accept null value for text in an email 2023-02-01 13:44:27 +08:00
Hongbo Wu
83c8166b3b Unescape HTML entities in annotation to avoid showing escaped characters 2023-02-01 12:13:00 +08:00
Jackson Harper
749df941f8 Use ilike for recent email query 2023-01-28 16:18:06 +08:00
Jackson Harper
7476878254 Add more logging for recent email lookups 2023-01-28 15:05:45 +08:00
Jackson Harper
6a50e85759 Save the parsed email address as the to field in received_emails 
The unparsed addresses can look something like this:

"jacksonh-XXXX@inbox.omnivore.app" <jacksonh-XXXX@inbox-demo.omnivore.app>

but we want to join on this field to the NewsletterEmail.address
field which will look like `jacksonh-XXXX@inbox.omnivore.app`
 2023-01-28 13:53:23 +08:00
Jackson Harper
15782fc5ce Add some analytics when creating reports 2023-01-26 19:32:41 +08:00
Jackson Harper
506568e0c2
Merge pull request #1667 from omnivore-app/recent-emails 
recent emails
 2023-01-25 14:22:00 +08:00
Hongbo Wu
a682d87aa6 Add tests for recent emails resolver 2023-01-19 12:11:34 +08:00
Hongbo Wu
83c62c65a3 Add test for saving received email 2023-01-19 10:44:08 +08:00
Hongbo Wu
318a6557a5 Email us if a recent email marked as library item 2023-01-19 09:55:49 +08:00
Hongbo Wu
7dbf8662fc Fix test 2023-01-19 09:38:21 +08:00
Hongbo Wu
8f9c317dd1 Save received email before publishing data 2023-01-18 22:35:07 +08:00
Jackson Harper
08048405a1 Correct typo in doc about labels 2023-01-18 22:07:39 +08:00
Hongbo Wu
36d2ac041a Fix error of getting subscriptions without newsletter email by inner join them 2023-01-18 18:42:10 +08:00
Hongbo Wu
d60890d154 Add pubsub to save received email 2023-01-18 17:13:11 +08:00
Hongbo Wu
60e107964f Fix typo 2023-01-18 15:00:34 +08:00
Hongbo Wu
96f727a165 Add mark a recent email as library item api 2023-01-18 15:00:34 +08:00
Hongbo Wu
62a3b48e00 Add list recent emails api 2023-01-18 15:00:34 +08:00
Hongbo Wu
3f373ab3b6 Save received email 2023-01-18 15:00:34 +08:00
Hongbo Wu
2da4ca5da8 Add saveReceivedEmail service 2023-01-18 15:00:34 +08:00
Hongbo Wu
b7d732df7c Add received_email entity 2023-01-18 15:00:34 +08:00
Hongbo Wu
f01ab3805a Left join active subscriptions of a newsletter email to get the right count 2023-01-18 14:37:54 +08:00
Hongbo Wu
e94f6de466 Delete subscription if unsub 2023-01-18 14:36:23 +08:00
Jackson Harper
786ad35da9
Merge pull request #1635 from omnivore-app/fix/content-display-reports 
Fix creation of content display report
 2023-01-13 11:37:22 +08:00
Hongbo Wu
6572f718c9
Merge pull request #1648 from omnivore-app/fix/puppeteer-parse 
fix/puppeteer parse
 2023-01-13 10:59:09 +08:00
Hongbo Wu
c647aabeaa Save newsletter_email_id in subscription instead of the whole entity 2023-01-12 22:15:42 +08:00
Hongbo Wu
7bfd36e308 Fix readability not run in puppeteer-parse 2023-01-12 18:33:47 +08:00
Jackson Harper
824899760a Update some style, add Matter archive importer tool 2023-01-12 14:00:46 +08:00
Jackson Harper
337b723d24 Fix creation of content display report 2023-01-11 10:51:46 +08:00
Hongbo Wu
43e813a73d Add unique constraint on user_id and name to subscription entity 2023-01-09 15:12:26 +08:00
Hongbo Wu
892a9ef30d Add subscriptionCount and createdAt in NewsletterEmail type 2023-01-09 12:24:45 +08:00
Hongbo Wu
3c8416ca1b
Merge pull request #1620 from omnivore-app/fix/sign-up 
Log email content if Sendgrid api key not set
 2023-01-06 15:39:33 +08:00
Hongbo Wu
9762b14ebc Check if the env is local 2023-01-06 15:28:45 +08:00
Jackson Harper
3520428ff0 Get the auth token from cookie in the user_router 
This is called through Cloud Task so .authorization is actually
Cloud Task's auth token.
 2023-01-06 10:29:36 +08:00
Hongbo Wu
d86a6038ea Log email content if Sendgrid api key not set 2023-01-06 09:54:02 +08:00
Jackson Harper
88b6fcfb5d Use cookie auth instead of header as GCP uses the Authorization header 2023-01-05 20:17:43 +08:00
Hongbo Wu
24763669b3
Merge pull request #1616 from omnivore-app/html-to-markdown 
html to markdown
 2023-01-05 19:30:19 +08:00
Hongbo Wu
e28c9b10bb Fetch markdown content if specified in search API 2023-01-05 17:17:09 +08:00
Jackson Harper
b0938ea8e2
Merge pull request #1612 from omnivore-app/feat/email-after-import 
Send email to user after a bulk import completes
 2023-01-05 13:39:49 +08:00
Hongbo Wu
7b749f974d Convert HTML to markdown if specified 2023-01-05 12:12:48 +08:00
Jackson Harper
89454d8b59
Merge pull request #1610 from omnivore-app/fix/reduce-logging 
Remove some places we are logging too much data causing sentry failures
 2023-01-05 11:19:30 +08:00
Jackson Harper
cfa3b5de89 Remove unneeded linting rules 2023-01-05 10:25:10 +08:00
Jackson Harper
7b2bc21b16 Add cors config, handle exception response 2023-01-04 21:45:41 +08:00
Jackson Harper
d89a307aca Send email to user after a bulk import completes 2023-01-04 18:15:26 +08:00
Jackson Harper
2390f2d716 Cast data to NewsletterMessage 2023-01-04 16:24:41 +08:00
Jackson Harper
e3ed7d0250 More logging changes 2023-01-04 11:23:13 +08:00
Jackson Harper
b438d42f38 Remove some unneeded console.log calls 2023-01-04 11:18:37 +08:00
Jackson Harper
8edd94bdb5 Remove some places we are logging too much data causing sentry failures 2023-01-04 11:07:54 +08:00
Jackson Harper
f939786c2a Reserve some names for tools and landing pages 2023-01-03 20:55:30 +08:00
Jackson Harper
0edd91057e Update user interface for file import tool 2023-01-03 17:49:07 +08:00
Jackson Harper
9314c3d8f1 Add uploadImportFile API method 
Add uploadImportFile API method

Fix prefix, counting max files uploaded

Add resolver types

Basic web ui for the uploader interface

Allow selecting type when uploading import files
 2023-01-03 10:01:59 +08:00
Hongbo Wu
2c39916441 Add wordsCount to article schema 2023-01-02 16:49:11 +08:00
Hongbo Wu
e0b9a6689a Not raise error when there is conflict of setting labels in highlights 2022-12-30 11:22:41 +08:00
Hongbo Wu
e0437008d7 Fix not updating/deleting the right label in the highlight 2022-12-29 22:40:31 +08:00
Hongbo Wu
264ebef46e Fix not setting labels to the correct highlight in a page 2022-12-29 21:34:45 +08:00
Hongbo Wu
7c39db207b Replace createArticle with savePage in puppeteer-parse service 2022-12-28 10:15:05 +08:00
Hongbo Wu
e22d209721 Fix test 2022-12-28 09:16:44 +08:00
Hongbo Wu
e9b31e375f Update savePage API to accept parsed content in the param 2022-12-27 21:59:31 +08:00
Hongbo Wu
302ea2023f Add obsidian to corsConfig 2022-12-23 10:29:43 +08:00
Jackson Harper
34029bee7e Update getting started guide, improve section on newsletter forwards, fix link to Logseq guide 2022-12-14 17:59:32 +08:00
Hongbo Wu
83aa2093eb Still return admins if not allowed to see members 2022-12-13 19:05:07 +08:00
Hongbo Wu
943b20f21e Only recommend page if the user is allowed to post 2022-12-13 19:00:49 +08:00
Hongbo Wu
5ef4e8479d Create one rule for all pages in the group to have the label and another rule for the recommended page to send a notification 2022-12-13 16:35:39 +08:00
Hongbo Wu
a340699240 add comment 2022-12-13 12:15:41 +08:00
Hongbo Wu
918ada2100 only recommend highlights created by the user 2022-12-13 12:12:20 +08:00
Hongbo Wu
25842270d7 Update the rule to only trigger on page created event 2022-12-13 12:04:17 +08:00
Hongbo Wu
4247835dc5
Merge pull request #1537 from omnivore-app/group-rules 
group rules
 2022-12-13 09:54:20 +08:00
Hongbo Wu
f5650acb69 Return canPost and canSeeMembers and topics and description in type group 2022-12-12 22:57:58 +08:00
Hongbo Wu
54ceeb326a Add canPost and canSeeMembers to type Group 2022-12-12 22:40:19 +08:00
Hongbo Wu
3e79b86179 Allow search recommendations with wildcard 2022-12-12 22:09:25 +08:00
Hongbo Wu
1338b1fc10 Remove subscription from shared page 2022-12-12 17:53:58 +08:00
Hongbo Wu
70ed3dccb7 Add analytic events for create/join/leave group actions 2022-12-12 17:49:00 +08:00
Hongbo Wu
5650971042 Do not save recent searches for rule based queries 2022-12-12 17:40:40 +08:00
Hongbo Wu
940c449880 revert for update page 2022-12-12 10:17:34 +08:00
Hongbo Wu
3347e81e39 Fix rule action not triggered due to the page index not being refreshed after page creation 2022-12-10 13:27:32 +08:00
Hongbo Wu
845235268d
Merge pull request #1525 from omnivore-app/recommend-api 
Add recommendation to the original page too
 2022-12-09 21:40:26 +08:00
Hongbo Wu
f4fa9cb15b do the same for merge highlight 2022-12-09 15:29:15 +08:00
Hongbo Wu
5fc18770ad Upsize max highlight char length to 6000 2022-12-09 15:24:10 +08:00
Hongbo Wu
181b401030 Update new recommendation notification message 2022-12-09 10:56:08 +08:00
Hongbo Wu
1f54db79fb Add recommendation to the original page too 2022-12-09 10:10:02 +08:00
Hongbo Wu
b8e7b15805 Fix generating colors with less than 6 digits 2022-12-08 19:01:54 +08:00
Hongbo Wu
04cb360c2f Fix some issue 2022-12-08 18:50:36 +08:00
Hongbo Wu
b942303b18 Create a rule after creating or joining a group 2022-12-08 18:35:22 +08:00
Hongbo Wu
9f260c1442 Create a label for new group if not exists 2022-12-08 18:20:58 +08:00
Hongbo Wu
84d8a9bb67 lint 2022-12-08 16:36:17 +08:00
Hongbo Wu
ff3e8aa29e Add delete permission to group and membership tables 2022-12-08 16:29:46 +08:00
Hongbo Wu
6830da995f Add leaveGroup API 2022-12-08 15:36:28 +08:00
Hongbo Wu
4c92af1856 Allow recommended With Highlights 2022-12-08 12:51:01 +08:00
Hongbo Wu
ef13cbf39a Recommend highlights 2022-12-08 12:33:05 +08:00
Hongbo Wu
62a3f6d1d8 Add recommendHighlights API 2022-12-08 11:37:35 +08:00
Jackson Harper
b150083358 Remove debug 2022-12-07 11:53:48 +08:00
Jackson Harper
533a4ae65c Update recommendedBy query 2022-12-06 21:23:19 +08:00
Jackson Harper
a8df439297 Rename recommendedBy to recommendations 2022-12-06 21:18:21 +08:00
Jackson Harper
0892d9e7e6 Add user and note to Recommendation type 2022-12-06 17:44:53 +08:00
Jackson Harper
8ac1afe277 Use cookie auth instead of header to not interfere with GCPs auth mechanisms 2022-12-05 18:41:08 +08:00
Jackson Harper
e555cc733a More descriptive error messages 2022-12-05 18:40:33 +08:00
Hongbo Wu
c49db571cd Add search by recommendedBy group name 2022-12-05 14:53:29 +08:00
Hongbo Wu
bf18c3d3c7 Add recommendedBy in search result 2022-12-05 14:27:46 +08:00
Hongbo Wu
e6b56e6fba Rename groups service 2022-12-05 14:12:04 +08:00
Hongbo Wu
714f0f1f63 Add joinGroup API 2022-12-05 12:47:38 +08:00
Hongbo Wu
27d9e8464d Add unique constraint on user_id and group_id to group_membership table 2022-12-05 12:45:54 +08:00
Hongbo Wu
6b65c386a0 Add authentication to the request header 2022-12-02 21:15:33 +08:00
Hongbo Wu
7707f100f2 Add recommend gql api 2022-12-02 19:45:26 +08:00
Hongbo Wu
0e3d1ced46 Add recommend rest api to handler tasks 2022-12-02 19:10:41 +08:00
Hongbo Wu
5fd137d23e Add recommendation cloud task 2022-12-02 18:49:13 +08:00
Hongbo Wu
a7d0525d2e Add recommendPage helper function 2022-12-02 18:26:16 +08:00
Hongbo Wu
b19b04bbe6 Add recommendedBy nested field in page index 2022-12-02 15:24:37 +08:00
Hongbo Wu
93c6c4202a Max number of group per user is 3 2022-12-02 12:36:18 +08:00
Hongbo Wu
da3d9cf76b Add groups api 2022-12-02 12:32:26 +08:00
Hongbo Wu
7285ebb940 Add createGroup API 2022-12-01 18:41:00 +08:00
Hongbo Wu
7f04a381a1 Add is_admin field to group_membership table 2022-12-01 17:47:12 +08:00
Hongbo Wu
7c8b057f85
Merge pull request #1486 from omnivore-app/fix/site-name-extraction 
Fix site_name not saved when a page is saved by extensions
 2022-12-01 11:28:51 +08:00
Hongbo Wu
0693a6c2ff
Merge pull request #1487 from omnivore-app/saved-filters-api 
saved filters api
 2022-12-01 11:25:50 +08:00
Hongbo Wu
0eebfe923a Add position in filter table and move filter api 2022-12-01 10:26:32 +08:00
Hongbo Wu
0487d8325c Add saved filter api 2022-11-30 21:41:59 +08:00
Hongbo Wu
1d514a0530 Update table name 2022-11-30 20:39:09 +08:00
Hongbo Wu
4d845ef413 Add search_filter table and entity 2022-11-30 20:38:30 +08:00
Hongbo Wu
bb839960f1 Fix site_name not saved when a page is saved by extensions 2022-11-30 17:46:59 +08:00
Hongbo Wu
86dccc0d44 Update savedAt in update page api 2022-11-30 13:17:50 +08:00
Hongbo Wu
28025db959 Add labels in highlight type in api response 2022-11-25 17:12:47 +08:00
Jackson Harper
e4e21ddc3a
Merge pull request #1462 from omnivore-app/add-order-in-updatesince-api 
Add sort in updatesince api and default sorting by updatedAt in descending order
 2022-11-25 13:18:05 +08:00
Hongbo Wu
64fd0cfdc6 Add sort in updatesince api and default sorting by updatedAt in descending order 2022-11-25 11:44:55 +08:00
Hongbo Wu
319c0c4bcd Allow changing byline in update api 2022-11-24 16:25:27 +08:00
Hongbo Wu
eb345ba393
Merge pull request #1449 from omnivore-app/more-filters-in-rules 
Use search API to filter page in rule engine
 2022-11-24 15:30:18 +08:00
Hongbo Wu
3f24d9a99e Add includes filter to search by ids in search api 2022-11-24 14:10:13 +08:00
Hongbo Wu
05ecac7c52 Fix tests 2022-11-24 08:59:21 +08:00
Hongbo Wu
f473c18939 Add deleteRule api 2022-11-23 09:52:15 +08:00
Hongbo Wu
ba730bdc3e Use default application credential 2022-11-22 13:00:01 +08:00
Hongbo Wu
2ae3226bdb Add sendNotification api 2022-11-22 12:48:17 +08:00
Hongbo Wu
9344fae1b9 Add title and image url in notification 2022-11-22 10:50:38 +08:00
Hongbo Wu
6b772c9248 Disable push notification reminders 2022-11-22 10:20:52 +08:00
Jackson Harper
080a1982b2 Remove unused import 2022-11-21 22:39:10 +08:00
Jackson Harper
1e4658b771 Consistent function declaration syntax 2022-11-21 22:39:10 +08:00
Jackson Harper
c0d5d12ec0 Add missing file 2022-11-21 22:39:10 +08:00
Jackson Harper
a3f477f57a Linting fix 2022-11-21 22:39:10 +08:00
Jackson Harper
9741f6b12d Add highlights to newly saved pages if they contain highlight markers 2022-11-21 22:39:10 +08:00
Hongbo Wu
f358094bbf Add get device tokens api 2022-11-21 17:03:12 +08:00
Hongbo Wu
88416bc178 Add get rules api 2022-11-21 15:03:15 +08:00
Hongbo Wu
dc92fe9c52 Create rule handler cloud run 2022-11-19 11:15:26 +08:00
Hongbo Wu
075c1b4d14 Fix send notification failure 2022-11-18 17:43:21 +08:00
Hongbo Wu
ecc217deef Fix test 2022-11-18 16:25:22 +08:00
Hongbo Wu
06c30e4f40 Add notification cloud run 2022-11-18 16:13:49 +08:00
Hongbo Wu
e53d5036dc Add notification endpoint 2022-11-18 14:33:27 +08:00
Hongbo Wu
fbe4c471cd Add pubsub subscription to trigger rule actions 2022-11-18 14:00:10 +08:00
Hongbo Wu
50fe92ac85 Add setRule resolver 2022-11-17 18:53:05 +08:00
Hongbo Wu
845e932d93 Store actions as a json in db 2022-11-17 16:21:43 +08:00
Hongbo Wu
23c527c345 Add rule entity 2022-11-17 15:04:44 +08:00
Hongbo Wu
60027dd670 Add rules table migration sql 2022-11-17 14:50:57 +08:00
Hongbo Wu
71348ab492 Remove trigger type in gql schema 2022-11-17 14:50:57 +08:00
Hongbo Wu
775180d785 Add rules api gql schema 2022-11-17 14:50:57 +08:00
Hongbo Wu
6a9a5c3870
Merge pull request #1408 from omnivore-app/fix/use-must-clause 
Replace filter clause with must clause in the search query in elastic
 2022-11-14 11:32:33 +08:00
Hongbo Wu
3f6a3b36ff Replace default voice to Larry for pre-generating 2022-11-11 18:34:44 +08:00
Hongbo Wu
2301f44848 Replace filter clause with must clause in the search query in elastic 2022-11-11 18:31:47 +08:00
Hongbo Wu
a78074b0da
Merge pull request #1406 from omnivore-app/set-voice-in-user-personalization 
set voice in user personalization
 2022-11-11 16:57:49 +08:00
Hongbo Wu
9872355056 Allow apollo sandbox access in local env only 2022-11-11 16:18:11 +08:00
Hongbo Wu
917396c976 Upsert user personalization in db 2022-11-11 16:11:09 +08:00
Hongbo Wu
67b26be547 Allow apollo sandbox access 2022-11-11 15:24:54 +08:00
Jackson Harper
83dbb54896 Allow filtering by author 2022-11-11 15:01:14 +08:00
Hongbo Wu
477b5d7d24 Fix typeorm issue 2022-11-11 14:53:46 +08:00
Hongbo Wu
ac21e4e6c8 Set voice and secondary voice in user personalization 2022-11-11 14:08:28 +08:00
Hongbo Wu
ce18d33ae1 Change queue name 2022-11-11 10:14:46 +08:00
Hongbo Wu
d58fb63c87 Pre-fetch realistic voice only if user is opted in 2022-11-10 18:49:52 +08:00
Hongbo Wu
b0938ee96c Pre-fetch realistic voice only if user is opted in 2022-11-10 18:48:00 +08:00
Hongbo Wu
69d6b60adc Extend expiration of the jwt token to 1 year 2022-11-10 17:42:25 +08:00
Hongbo Wu
399e31534d Update jwt token 2022-11-09 22:48:33 +08:00
Hongbo Wu
c89ffdeaf9 Fix some tests 2022-11-09 18:14:15 +08:00
Hongbo Wu
63de1c3359 Add optInFeature API 2022-11-09 17:35:54 +08:00
Hongbo Wu
98f59c50f0 Remove token in features table 2022-11-09 11:32:08 +08:00
Hongbo Wu
c2f4ba4a57 Add features entity 2022-11-08 18:14:23 +08:00
Hongbo Wu
398e242131 Add features table 2022-11-08 17:30:03 +08:00
Hongbo Wu
f2a69d06c4 Change default rate to 1.1 2022-11-08 16:53:35 +08:00
Hongbo Wu
ddd1e84686 Add language and rate in the cloud task params 2022-11-08 16:22:23 +08:00
Hongbo Wu
769e8c5f96 Add default language and rate 2022-11-08 15:41:20 +08:00
Hongbo Wu
3ffd7739bb Default voice is Larry 2022-11-08 14:23:48 +08:00
Hongbo Wu
44987072ee Get voice from user personalization 2022-11-08 13:08:05 +08:00
Hongbo Wu
271280f4fc Fix not correctly getting audio file 2022-11-08 12:56:52 +08:00
Hongbo Wu
e62765c3e6 Save steaming data in GCS 2022-11-08 11:24:47 +08:00
Hongbo Wu
1af5af0e97 Get realistic voice id from user personalization table 2022-11-08 10:21:27 +08:00
Hongbo Wu
2e47b0879c Convert HTML to utterances 2022-11-07 19:30:20 +08:00
Hongbo Wu
d5583455ca Forward text if html is not found in the email 2022-11-01 12:47:23 +08:00
Hongbo Wu
028b212ecb Fix readwise api error when image_url is empty by removing image_url from api payload 2022-10-31 12:45:49 +08:00
Hongbo Wu
e87b81c5c8 Use order instead of page to indicate highlight location in Readwise 2022-10-28 17:44:14 +08:00
Hongbo Wu
1c31e5fc15 Skip sync with Readwise if no highlights 2022-10-28 15:05:16 +08:00
Hongbo Wu
bad6904909
Merge pull request #1355 from omnivore-app/fix/sync-integration 
Use the correct data format in the pub/sub queue
 2022-10-27 14:15:01 +08:00
Hongbo Wu
3446bc3555 Use the correct data format in the pub/sub queue 2022-10-27 14:01:32 +08:00
Hongbo Wu
f0dc9b7d13 Fix tests 2022-10-27 12:45:32 +08:00
Hongbo Wu
2169ade48f Persist integration in db before creating cloud task to sync with integrations 2022-10-27 11:32:56 +08:00
Hongbo Wu
614c6a2b95
Merge pull request #1347 from omnivore-app/fix/puppeteer-timeout 
Use auth token to identify client request in rate limiter
 2022-10-25 17:21:13 +08:00
Hongbo Wu
57ba9afbfa Remove debugging 2022-10-25 13:17:57 +08:00
Hongbo Wu
c9ea622461 Use auth token to identify client request in rate limiter 2022-10-25 12:42:32 +08:00
Hongbo Wu
5e68e30d5b Add highlightPositionPercent and highlightPositionAnchorIndex in highlight api 2022-10-25 10:24:30 +08:00
Hongbo Wu
a2f31c943f Fix tests 2022-10-20 11:31:39 +08:00
Hongbo Wu
e1de55bea0 Delete highlights, labels and other metadata when soft delete pages 2022-10-20 11:16:49 +08:00
Jackson Harper
d5c7aa2b8f Fix linting 2022-10-19 17:36:58 +08:00
Jackson Harper
0ad8558e92 Add the iOS Saving Links article as a default read for new Omnivore users on iOS 
This also re-orders things a bit, making sure the getting started
guide is always the topmost in a user's library.
 2022-10-19 17:22:30 +08:00
Jackson Harper
d34e5aff04 Change the order of the new user welcome articles 
In the UI it will appear:

- Getting Started
- Power of Read it Later
- Organize with Labels
- (optional) iOS/Android/Web
 2022-10-18 17:35:00 +08:00
Hongbo Wu
a7e92addb0 Create 128 * 128 proxy image for the site icon 2022-10-18 15:32:30 +08:00
Hongbo Wu
22e92256eb Fetch subscription's favicon if the site icon is base64 encoded image 2022-10-18 15:32:30 +08:00
Jackson Harper
5312c31ea7 Dont crash if a popular read is not available on signup 2022-10-17 20:45:04 +08:00
Jackson Harper
08a1d4af91 Update the Getting started guide for new users 2022-10-17 17:11:25 +08:00
Hongbo Wu
496b28c9ee Return deleted article in the updatesSince API response 2022-10-14 13:50:53 +08:00
Hongbo Wu
14c87a1240
Merge pull request #1298 from omnivore-app/feature/add-popular-read-when-signup 
feature/add popular read when signup
 2022-10-13 13:13:13 +08:00
Hongbo Wu
19d0d314db Fix tests 2022-10-12 18:05:17 +08:00
Hongbo Wu
7a48cccac9 Save client in request context 2022-10-12 14:30:39 +08:00
Hongbo Wu
9496827c6b Add popular reads to new users based on client 2022-10-12 13:59:08 +08:00
Hongbo Wu
4b7950458c Remove Redis client in api 2022-10-12 13:40:16 +08:00
Hongbo Wu
bccb949fce Save advanced search terms in search history too 2022-10-12 13:35:01 +08:00
Hongbo Wu
d4f32f7bdd Save search term in search history and update timestamp 2022-10-12 13:27:19 +08:00
Hongbo Wu
8f96c167f8 Save search term in search history 2022-10-12 12:30:22 +08:00
Hongbo Wu
1725f96d82 Add recentSearches API implementation and test 2022-10-12 12:20:48 +08:00
Hongbo Wu
fd1d43c104 Add search history repo methods 2022-10-12 10:49:07 +08:00
Hongbo Wu
ff12675713 Make user_id and term unique 2022-10-12 10:37:29 +08:00
Hongbo Wu
ab86450864 Rename X-Device to X-OmnivoreClient in the request headers 2022-10-12 10:27:39 +08:00
Hongbo Wu
2ba9247a98 Device info expires in 10 mins 2022-10-11 19:03:33 +08:00
Hongbo Wu
2210a38982 Set device of the request in Redis 2022-10-11 18:47:19 +08:00
Hongbo Wu
af9147d9d3 Add iOS popular reads to new iOS users 2022-10-11 16:07:49 +08:00
Hongbo Wu
606183829a Add popular reads to new users 2022-10-11 15:44:53 +08:00
Hongbo Wu
a5c0ef19ec Add recentSearches API GQL schema 2022-10-11 14:53:48 +08:00
Hongbo Wu
4e14518189 Replace keyword with term 2022-10-11 11:26:18 +08:00
Hongbo Wu
cd13e5874a Create search_history table and entity class 2022-10-10 17:44:55 +08:00
Hongbo Wu
5d70d04154 If the saved tweet is a reply, we need to get the referenced tweet 2022-10-10 15:12:41 +08:00
Hongbo Wu
49ed8e279b Calls preParseContent function in content-handler package before parsing content 2022-10-07 16:56:38 +08:00
Hongbo Wu
679977805f Remove custom content-handler in packages/api 2022-10-07 16:56:38 +08:00
Hongbo Wu
8942398092 Change GET to HEAD 2022-09-30 12:52:09 +08:00
Hongbo Wu
9841ce7f8e Remove parsing newsletter emails from forwarded emails 2022-09-30 12:51:23 +08:00
Hongbo Wu
b00a516737 Add other newsletter handlers 2022-09-30 12:51:23 +08:00
Hongbo Wu
57ca5ed6f8
Merge pull request #1246 from omnivore-app/feature/subscription-icon 
feature/subscription icon
 2022-09-28 22:10:23 +08:00
Hongbo Wu
2f6b26d21d Use GET request as some host do not allow HEAD requests 2022-09-28 20:18:40 +08:00
Hongbo Wu
a55ad135fd Set 5s for the HEAD request timeout 2022-09-28 19:26:42 +08:00
Hongbo Wu
d6974f9e5f Remove unused icon 2022-09-28 19:14:14 +08:00
Hongbo Wu
a6795b380a Fetch favicon from url 2022-09-28 19:02:29 +08:00
Hongbo Wu
4aceac5f04 Revert "Get favicon from URL if not found from doc" 
This reverts commit 7306d65642.
 2022-09-28 18:25:11 +08:00
Hongbo Wu
7306d65642 Get favicon from URL if not found from doc 2022-09-28 17:18:00 +08:00
Hongbo Wu
ee3fec9d96
improve text to speech dependency (#1250) 
* Generate .d.ts files from TypeScript for text-to-speech project

* Remove module export for htmlToSpeechFile

* Include test
 2022-09-28 11:48:06 +08:00
Hongbo Wu
66d37e31b1 Use siteIcon 2022-09-27 21:41:01 +08:00
Hongbo Wu
6220d5d4af Return subscription icon proxy url in the list subscriptions api response 2022-09-27 15:35:06 +08:00
Hongbo Wu
d42d4ace98 Use parsed image as subscription icon 2022-09-27 15:21:18 +08:00
Hongbo Wu
5d62d92974 Save icon in the subscription table 2022-09-27 12:12:47 +08:00
Hongbo Wu
94719a5cce Add icon to the subscription entity 2022-09-27 12:00:13 +08:00
Hongbo Wu
710243e94e Allow language in the query to override the page language in the speech file 2022-09-23 14:19:29 +08:00
Hongbo Wu
0239103fc4 Remove the table with text content less than 20 because these are ads 2022-09-22 22:16:03 +08:00
Hongbo Wu
56e3ccadf0 Add test case for Axios AM newsletter 2022-09-21 12:16:13 +08:00
Hongbo Wu
2c3b8535d8 Fix one morning brew newsletter parsing error by only keeping market info table in the original newsletter 2022-09-20 17:12:13 +08:00
Hongbo Wu
b995dc0500 Stop throwing error and return empty speech if there is no content in the article 2022-09-19 17:36:58 +08:00
Hongbo Wu
b03f7ebeb8 Add handler to pre-process morning brew newsletters 2022-09-15 16:21:09 +08:00
Hongbo Wu
d795fef8a7 Add handler to pre-process morning brew newsletters 2022-09-15 16:20:57 +08:00
Hongbo Wu
fa731acd74 Create new newsletter if old one is deleted 2022-09-14 19:14:58 +08:00
Hongbo Wu
e86a107a9a Bump up image size to 160*160 2022-09-12 17:05:31 +08:00
Hongbo Wu
a01a10e3f5 Put title the first utterance 2022-09-09 12:10:29 +08:00
Hongbo Wu
06218ef5e0 Return image proxy url in Article function_resolver 2022-09-08 17:55:09 +08:00
Hongbo Wu
6350a3d53e Make idx string type 2022-09-08 11:33:29 +08:00
Hongbo Wu
8e8a6dccd0 Change rate to string type 2022-09-07 22:55:07 +08:00
Hongbo Wu
9a62a3c31e Add pageId to speech-file 2022-09-07 22:51:44 +08:00
Hongbo Wu
3cf3d40482 Replace speech-file with speech in the route 2022-09-07 22:48:40 +08:00
Hongbo Wu
aeb7e47d68 Make voice, secondaryVoice and priority query params 2022-09-07 18:42:46 +08:00
Hongbo Wu
f1b379b96e Convert html to speech file and return 2022-09-07 18:00:32 +08:00
Hongbo Wu
96cbdc9f30 Replace wordsCount with wordCount 2022-09-07 15:16:16 +08:00
Hongbo Wu
5781d385ad Return utterances in the API if output format is ssml 2022-09-07 15:16:16 +08:00
Hongbo Wu
d4dcec75d5 Return 0 for wordsCount if error 2022-09-07 15:16:16 +08:00
Hongbo Wu
97e560c443 Update wordsCount in page 2022-09-07 15:16:16 +08:00
Hongbo Wu
795ca47414 Save wordsCount when page is created 2022-09-06 10:19:58 +08:00
Hongbo Wu
d00391b593 Add wordCount to Page doc in elastic 2022-09-06 10:03:36 +08:00
Hongbo Wu
9826262ef4 Return an array of SSMLs instead of ssmlItems 2022-09-02 15:33:00 +08:00
Hongbo Wu
3953a9098e Send SSML items in the synthesis API 2022-09-02 15:33:00 +08:00
Hongbo Wu
7353c328d9 Remove unused text-to-speech code 2022-08-29 22:29:40 +08:00
Hongbo Wu
d085c86bb6 Enqueue text to speech tasks 2022-08-29 22:23:07 +08:00
Hongbo Wu
8c8734d153 Re-enable auto synthesis in the backend 2022-08-29 22:09:09 +08:00
Hongbo Wu
a94dca8932 Prioritize synthesis over playing audio over pre-fetching 2022-08-29 15:45:21 +08:00
Hongbo Wu
4672b520e9
Merge pull request #1148 from omnivore-app/feature/separate-queues-for-synthesis 
feature/separate queues for synthesis
 2022-08-29 13:43:53 +08:00
Hongbo Wu
a19ec8efea Hardcode queue name 2022-08-29 13:20:39 +08:00
Satindar Dhillon
dfea982df7
Merge pull request #1146 from omnivore-app/feature/android-apple-login-redirect 
Android Apple Login Endpoint
 2022-08-28 21:17:32 -07:00
Hongbo Wu
2e4cb7f4b0 Fix using wrong taskHandlerUrl for low priority task 2022-08-29 12:01:17 +08:00
Hongbo Wu
4af340f30e Add the priority to request param and use the correct queue accordingly 2022-08-29 11:49:10 +08:00
Hongbo Wu
f903a92245 Use text-to-speech queue 2022-08-29 11:47:42 +08:00
Hongbo Wu
e976c9e169 Add text-to-speech queue name and location to env 2022-08-29 11:47:01 +08:00
Satindar Dhillon
3f6ade9614 verify apple token with both audience names used for the app 2022-08-28 09:55:41 -07:00
Satindar Dhillon
a720d5a5d5 set cors config for android web redirect 2022-08-28 09:48:39 -07:00
Satindar Dhillon
3d16b1d0b4 add redirect uri for android apple login 2022-08-28 09:36:30 -07:00
Hongbo Wu
7a9eca2622 Update text-to-speech endpoint 2022-08-26 17:53:26 +08:00
Hongbo Wu
5201f5d360 Add RLS to speech table 2022-08-26 16:56:33 +08:00
Hongbo Wu
b2d49d1fbf Add dotenv 2022-08-26 16:30:09 +08:00
Hongbo Wu
5a1dbb594e Call backend api to update speech in db 2022-08-26 15:41:06 +08:00
Jackson Harper
933fa9f142 Use a promise 2022-08-25 20:07:13 +08:00
Jackson Harper
82fdad3c80 Disable the automatic speech trigger 2022-08-25 19:54:39 +08:00
Hongbo Wu
b337735c9d Do not throw error if failed to synthesize self-closing tags elements 2022-08-25 18:52:10 +08:00
Hongbo Wu
3945c2f887 Prevent synthesizing PDF 2022-08-25 17:01:29 +08:00
Hongbo Wu
3cea785336 Fix page id not get in auto-synthesize subscription 2022-08-25 16:09:35 +08:00
Hongbo Wu
7f83045c0c Fix a bug 2022-08-25 16:09:35 +08:00
Hongbo Wu
99a628f03f Fix tests 2022-08-25 16:09:35 +08:00
Hongbo Wu
a477560626 Add a feature flag for text to speech beta test and if turned on, automatically synthesize new articles 2022-08-25 16:09:35 +08:00
Hongbo Wu
00d1696f23 use complimentary voice for blockquote, hardcoded for now 2022-08-25 16:09:35 +08:00
Hongbo Wu
4f8d71b7cc Add a pubsub subscription listening on the new article to automatically synthesize article 2022-08-25 16:09:35 +08:00
Hongbo Wu
9b0093e3d2 Add function to test if a page should be transcribed 2022-08-25 16:09:35 +08:00
Hongbo Wu
5622154e6c Update listenedAt in the API 2022-08-25 16:09:35 +08:00
Satindar Dhillon
2a273ef504 specify multiple aud values when verifying google tokens 2022-08-24 21:00:15 -07:00
Satindar Dhillon
ba1c9c970b use correct aud value when verifying google token from android 2022-08-23 09:26:47 -07:00
Hongbo Wu
45c1ba2d25 Update log 2022-08-22 16:45:10 +08:00
Hongbo Wu
5cfe0a14e9 Authorize user to get or create speech in the API 2022-08-22 16:44:05 +08:00
Hongbo Wu
cfdc236297 Validate token and authenticate user before setting Speech state to FAILED 2022-08-22 16:24:39 +08:00
Hongbo Wu
463f423cab Enqueue TTS task in the get speech of article API if speech not found 2022-08-22 16:13:34 +08:00
Hongbo Wu
efce6dad94 Stop queueing transcription after calling createArticle API 2022-08-22 16:12:27 +08:00
Hongbo Wu
476e43a1f8 Set speech state to FAILED if encountered error during transcription 2022-08-22 16:11:49 +08:00
Hongbo Wu
4852777f66 Throw error if synthesizing was cancelled 2022-08-22 16:10:49 +08:00
Hongbo Wu
2298635e56 Add function to set speech to failed state 2022-08-22 16:09:56 +08:00
Hongbo Wu
4ce7352013
Merge pull request #1108 from omnivore-app/fix/failed-to-convert-speech 
Fix some articles not converted to speech correctly
 2022-08-22 12:58:04 +08:00
Jackson Harper
965ffb3013
Merge pull request #1065 from omnivore-app/feature/android-login 
Android Login
 2022-08-22 12:41:04 +08:00
Hongbo Wu
85d8816e1e Replace li element with a sentence element when converting HTML to SSML 2022-08-22 12:39:06 +08:00
Satindar Dhillon
1115cf34d9 update google login client id for android 2022-08-19 19:39:41 -07:00
Satindar Dhillon
33a4ee39dc check for android source in api mobile sign up route 2022-08-19 18:53:51 -07:00
Hongbo Wu
dba749c506 Fix some articles not converted to speech correctly 2022-08-20 00:11:46 +08:00
Hongbo Wu
0cf689ce21 Fix not getting revue newsletter url by checking all the hyper links in the table tr td 2022-08-19 17:32:29 +08:00
Hongbo Wu
0a75c52a8c Fix getting script_exception when updating labels in highlights by checking null before updating 2022-08-19 14:36:58 +08:00
Hongbo Wu
9906e2153b Save correct mp3 and speech marks file name in the database 2022-08-19 10:04:13 +08:00
Satindar Dhillon
1b46739a0d add sign in option for android client on server 2022-08-18 18:42:18 -07:00
Hongbo Wu
a1171c7952 Catch authentication error 2022-08-18 19:24:38 +08:00
Hongbo Wu
576318fcbe Add authentication to speech svc endpoint 2022-08-18 19:24:38 +08:00
Hongbo Wu
401633eda6 Return signed url 2022-08-18 19:24:38 +08:00
Hongbo Wu
249a5cda4a Add bucket, audioFileName, speechMarksFileName, state to speech table 2022-08-18 19:24:38 +08:00
Hongbo Wu
4cf5b934eb Add validation to request params 2022-08-18 19:24:38 +08:00
Hongbo Wu
29061cf7b1 Remove &nbsp in SSML 2022-08-18 19:24:38 +08:00
Hongbo Wu
a4a8fa9241 Replace emphasis elements in HTML 2022-08-18 19:24:38 +08:00
Hongbo Wu
c79651202d Add synthesize SSML to speech and test 2022-08-18 19:24:38 +08:00
Hongbo Wu
dee94f7c93 Fix a bug for converting text to speech for articles with less than 5000 characters 2022-08-18 19:24:38 +08:00
Hongbo Wu
19b13d164d Fix a bug not getting user personalization 2022-08-18 19:24:38 +08:00
Hongbo Wu
83c1930378 Split text into chunks of 5000 characters in paragraph 2022-08-18 19:24:38 +08:00
Hongbo Wu
c9d26acb25 Fix typo 2022-08-18 19:24:38 +08:00
Hongbo Wu
02ba753375 Upload speech marks file to gcs too 2022-08-18 19:24:38 +08:00
Hongbo Wu
c482ad320b Add duration in logging 2022-08-18 19:24:38 +08:00
Hongbo Wu
b0bf4fc5ce Create speech while requesting 2022-08-18 19:24:38 +08:00
Hongbo Wu
301746d494 Add /:id/mp3 router to redirect request to speech MP3 URL 2022-08-18 19:24:38 +08:00
Hongbo Wu
c74f2bc3fd Enqueue text to speech task after creating article 2022-08-18 19:24:38 +08:00
Hongbo Wu
447e413605 Add function to parse HTML to SSML 2022-08-18 19:24:38 +08:00
Hongbo Wu
355e33b648 Test Mandarin 2022-08-18 19:24:38 +08:00
Hongbo Wu
e086a14ac2 Convert time to ms 2022-08-18 19:24:38 +08:00
Hongbo Wu
77570aa5ab Synthesize text to speech with azure API 2022-08-18 19:24:36 +08:00
Hongbo Wu
94f9dd9e6e Enqueue text to speech tasks 2022-08-18 19:23:41 +08:00
Hongbo Wu
4b42d013ca Convert text to OGG file 2022-08-18 19:23:41 +08:00
Hongbo Wu
0419472c2e Upload audio file with public access right 2022-08-18 19:23:41 +08:00
Hongbo Wu
dd332f5ee6 Add tests 2022-08-18 19:23:41 +08:00
Hongbo Wu
bca0f5ed59 Create Amazon Polly text to speech integration 2022-08-18 19:23:41 +08:00
Hongbo Wu
427cecf6b9 Update syncedAt after syncing 2022-08-09 09:51:31 +08:00
Hongbo Wu
134482a261 Add integration tests for integration router 2022-08-08 23:11:35 +08:00
Hongbo Wu
7ac45ca748 Add deleteIntegration API and integration tests 2022-08-08 18:34:13 +08:00
Hongbo Wu
4aa62d6031 Add list integrations API and integration tests 2022-08-08 18:12:44 +08:00
Hongbo Wu
0cf3f58258 Add integration tests for setIntegration API 2022-08-08 17:09:23 +08:00
Hongbo Wu
8070273dad Fix tests 2022-08-05 16:10:33 +08:00
Hongbo Wu
22ce59a198 Enqueue or delete sync task when create or disable integration 2022-08-05 16:09:26 +08:00
Hongbo Wu
5a8184da3a Add taskName in integrations table 2022-08-05 16:09:26 +08:00
Hongbo Wu
eb2ee5ab55 Add task to handle sync with integrations 2022-08-05 16:09:26 +08:00
Hongbo Wu
c0c58bf751 Only sync pages that were updated after syncedAt 2022-08-05 16:09:26 +08:00
Hongbo Wu
8e9d1fcc4f Update integration syncedAt if successful 2022-08-05 16:09:26 +08:00
Hongbo Wu
bd73da0571 Add syncedAt field in integrations table 2022-08-05 16:09:26 +08:00
Hongbo Wu
8aa7ccdd1e Add INTEGRATION_TASK_HANDLER_URL in env 2022-08-05 16:09:26 +08:00
Hongbo Wu
971bbfda7f Add integrations service router to sync pages with integrations 2022-08-05 16:09:26 +08:00
Hongbo Wu
24caac2ed5 Update integration updatedAt after syncing 2022-08-05 16:09:26 +08:00
Hongbo Wu
8f50285ad5 Allow search page by highlight or label id 2022-08-05 16:09:24 +08:00
Hongbo Wu
fa97c9511c Add Readwise create highlight API integration 2022-08-05 16:08:20 +08:00
Hongbo Wu
73bf695cf3 Add function to get highlight start location 2022-08-05 16:08:20 +08:00
Hongbo Wu
b8dbcdb2bd Add createHighlight API integration for Readwise 2022-08-05 16:08:20 +08:00
Hongbo Wu
dd2db71876 Add createIntegration API implementation 2022-08-05 16:08:20 +08:00
Hongbo Wu
e66d047d51 Add createIntegration API to the GQL schema 2022-08-05 16:08:20 +08:00
Hongbo Wu
db27ac7d59 Add integration entity 2022-08-05 16:08:20 +08:00
Hongbo Wu
977bd1deda Fix tests 2022-08-05 11:44:02 +08:00
Hongbo Wu
7c73982fe3 Add test for moving to the top 2022-08-04 21:39:45 +08:00
Hongbo Wu
a95f532e44 Add moveLabel api implementation 2022-08-04 21:38:36 +08:00
Hongbo Wu
b12607887e Add moveLable api in gql Schema 2022-08-04 21:38:05 +08:00
Hongbo Wu
31d8f72840 Sort labels by position order 2022-08-04 21:37:32 +08:00
Hongbo Wu
5e2e2a2151
Merge pull request #1044 from omnivore-app/fix/email-forwarding 
Fix not getting correct sender email address from forwarding email by Gmail rules
 2022-08-04 21:37:13 +08:00
Hongbo Wu
92d3ab0195
Merge pull request #1035 from omnivore-app/feature/sync-api 
feature/sync api
 2022-08-04 20:33:05 +08:00
Hongbo Wu
70deb348cf Fix not getting correct sender email address from forwarding email by Gmail rules 2022-08-04 18:50:23 +08:00
Satindar Dhillon
c076dd0f72 add mobile email sign up endpoint 2022-08-01 21:54:10 -07:00
Satindar Dhillon
c01d9a7021 server returns a pendingEmailVerification boolean in auth payload 2022-08-01 21:54:10 -07:00
Satindar Dhillon
6a100051ac add mobile email sign in route to api 2022-08-01 21:54:10 -07:00
Hongbo Wu
336d06375a Add updatesSince API resolver 2022-07-29 18:11:18 +08:00
Hongbo Wu
7ce183d6f8 Add soft delete of pages by updating page state to Deleted 2022-07-29 18:10:05 +08:00
Hongbo Wu
83f988bb35 Soft delete articles 2022-07-29 15:53:13 +08:00
Hongbo Wu
07c6214e88 Add DELETED state to the article 2022-07-29 14:08:35 +08:00
Hongbo Wu
b976b6098d Add updatesSince API in GQL schema 2022-07-29 12:11:48 +08:00
Jackson Harper
6d19c8c9bc Remove unused import 2022-07-28 11:43:16 -07:00
Jackson Harper
1525742b4e Linting fixes 2022-07-28 11:23:27 -07:00
Jackson Harper
565da42b46 When parsing newsletters with no url dont fetch the generated url 2022-07-28 11:05:17 -07:00
Hongbo Wu
3a120b8f47 Return empty name if name not found in from header 2022-07-28 10:37:28 +08:00
Hongbo Wu
c99c1db57e Add support to the case when from address is in Name <address> format 2022-07-28 10:26:15 +08:00
Hongbo Wu
0bdab51bbd
Merge pull request #1021 from omnivore-app/feature/save-from-email 
Save articles by sending emails to omnivore inbox
 2022-07-27 20:10:31 +08:00
Hongbo Wu
6f11ccacb1 Save article from forwarding emails 2022-07-27 12:15:28 +08:00
Hongbo Wu
d184ca8d04 Add function isProbablyArticle to test if a forwarded email contains an article to save 2022-07-27 12:15:26 +08:00
Jackson Harper
779ff1f020 Reset password is a directory now 2022-07-26 15:57:40 -07:00
Jackson Harper
65689ca538 Remove extra /auth in paths 2022-07-26 15:57:18 -07:00
Jackson Harper
3c442c06ba Set auth token cookie on success login 2022-07-26 15:56:40 -07:00
Jackson Harper
6cad41ec1e Linting fixes 2022-07-26 15:00:31 -07:00
Jackson Harper
769c9b23b0 Use SSO redirection to set both client and server cookies with email login 2022-07-26 14:48:24 -07:00
Jackson Harper
e939f12eb1 Remove unneeded branches 2022-07-26 14:28:39 -07:00
Jackson Harper
074ec781a2 Set auth cookie after reseting password 2022-07-26 12:46:13 -07:00
Jackson Harper
dc268a2216 Handle resetting passwords 2022-07-26 12:33:56 -07:00
Jackson Harper
acd36fe9af Move auth routes into /auth folder 2022-07-26 10:31:45 -07:00
Hongbo Wu
072b2dd9c5 Revert "Add Set-Cookie header" 
This reverts commit cb23613cdc.
 2022-07-26 14:12:37 +08:00
Jackson Harper
2d9f4a0194 Redirect to the verify-email informational page after sign up 2022-07-25 23:09:01 -07:00
Hongbo Wu
cb23613cdc Add Set-Cookie header 2022-07-26 14:06:11 +08:00
Hongbo Wu
dca9066f19 Continue fix RLS issue of updating user table 2022-07-26 13:41:02 +08:00
Hongbo Wu
29ed803ae5 Fix RLS issue of updating user table 2022-07-26 13:31:36 +08:00
Hongbo Wu
82da9e6ede use sendgrid template for confirmation and password-reset email 2022-07-25 15:14:01 -07:00
Hongbo Wu
8d5da15064 use template id stored in env 2022-07-25 15:14:01 -07:00
Hongbo Wu
fe180c5b50 add confirm-email and password-reset email template id in env 2022-07-25 15:14:01 -07:00
Hongbo Wu
bd77a7f8ee add test for reset password 2022-07-25 15:14:01 -07:00
Hongbo Wu
6699ec834d add router handler for reset password 2022-07-25 15:14:01 -07:00
Hongbo Wu
286d167769 add router handler for email reset password 2022-07-25 15:14:01 -07:00
Hongbo Wu
bab96aaa1e remove membership from user 2022-07-25 15:14:01 -07:00
Hongbo Wu
ea9d98aa95 make confirm-email a post api 2022-07-25 15:14:01 -07:00
Hongbo Wu
4c75260229 replace hard-coded email address with env var for the remaining files 2022-07-25 15:14:01 -07:00
Hongbo Wu
a87f75f319 add test for confirm-email router 2022-07-25 15:14:01 -07:00
Hongbo Wu
e4ea49c05f add confirm-email router 2022-07-25 15:14:01 -07:00
Hongbo Wu
9ea9bd9ea1 send confirmation email to pending user when login 2022-07-25 15:14:01 -07:00
Hongbo Wu
abb1a414c1 remove login gql resolver 2022-07-25 15:13:58 -07:00
Hongbo Wu
f7053622a5 remove signup gql resolver 2022-07-25 15:13:30 -07:00
Hongbo Wu
80639311d7 improve confirmation email format 2022-07-25 15:12:56 -07:00
Hongbo Wu
068684d16b add test for failing to send confirmation email when signup 2022-07-25 15:12:56 -07:00
Hongbo Wu
d284a3d302 fake sendMail in sign up test 2022-07-25 15:12:56 -07:00
Hongbo Wu
01c4bbad00 delete pending user if failed to send confirmation email 2022-07-25 15:12:56 -07:00
Hongbo Wu
b0a0020fc7 fix return false when sending failed 2022-07-25 15:12:56 -07:00
Hongbo Wu
4e11000ac0 add return type 2022-07-25 15:12:56 -07:00
Hongbo Wu
d187c6c993 use general sender for confirmation email 2022-07-25 15:12:55 -07:00
Hongbo Wu
d108806234 make sender address nullable env var 2022-07-25 15:12:55 -07:00
Hongbo Wu
b98f7c6ba2 reject with unknow reason if failed to send confirmation email 2022-07-25 15:12:55 -07:00
Hongbo Wu
1ea34bde62 send confirmation email to pending user 2022-07-25 15:12:55 -07:00
Hongbo Wu
7293469468 use sender email address in env 2022-07-25 15:12:55 -07:00
Hongbo Wu
bd5b289168 add verification token generation 2022-07-25 15:12:55 -07:00
Hongbo Wu
1346140a20 add sender email address in env 2022-07-25 15:12:55 -07:00
Hongbo Wu
ba73eec9e5 create pending user first 2022-07-25 15:12:55 -07:00
Hongbo Wu
35e060890e add status to user entity 2022-07-25 15:12:55 -07:00
Jackson Harper
c113c94567 Add param validation so we dont get undefined values passed to GQL 2022-07-21 17:43:15 -07:00
Jackson Harper
ff629b2fed
Merge pull request #992 from omnivore-app/fix/email-urls 
Use email-signup instead of email-registration as path to match naming
 2022-07-19 16:02:00 -07:00
Jackson Harper
e0a4478400
Merge pull request #954 from omnivore-app/feat/extension-commands 
Extension: support right click saveurl and shortcut key to save current tab
 2022-07-19 15:37:18 -07:00
Jackson Harper
d6a6b1de66 Use email-signup instead of email-registration as path to match naming 2022-07-19 13:02:28 -07:00
Hongbo Wu
65f28fe708 add revue.email to the revue newsletter attribute selectors 2022-07-18 10:44:39 +08:00
Hongbo Wu
345c020dea fix not saving loqseq weekly newsletter due to no url 2022-07-17 12:36:52 +08:00
Jackson Harper
4b8749ed43 Remove debug line 2022-07-16 12:40:54 -07:00
Jackson Harper
0bcb8a23c5 Use new API methods for saving 2022-07-16 12:17:58 -07:00
Jackson Harper
cb506fcf93 Remove extra quote from report string 2022-07-14 20:33:06 -07:00
Hongbo Wu
edd44fcafc fix get article test 2022-07-14 17:56:55 +08:00
Hongbo Wu
404798a01e use correct index name 2022-07-14 17:08:40 +08:00
Hongbo Wu
4eb55846fd create elastic index if not exists in testing 2022-07-14 16:53:23 +08:00
Hongbo Wu
5a8e2c40c7 fix delete users db permission error 2022-07-13 16:08:46 +08:00
Hongbo Wu
098771f566
Merge pull request #955 from omnivore-app/fix/duplicate-save 
check if page saved by adding links before creating by extension
 2022-07-13 11:57:20 +08:00
Hongbo Wu
01e95c3598 check if page saved by adding links before creating by extension 2022-07-13 11:34:20 +08:00
Hongbo Wu
cb1d64cd82 add siteName in api response 2022-07-13 10:57:54 +08:00
Hongbo Wu
970609c3b8 change size to first in query param 2022-07-13 10:46:03 +08:00
Hongbo Wu
931c519d44 typeahead searchs by siteName 2022-07-13 10:37:06 +08:00
Hongbo Wu
b68282fe97 rename typeahead to typeaheadSearch 2022-07-12 09:36:00 -07:00
Hongbo Wu
3a1f38a6f7 add typeahead api implementation 2022-07-12 09:36:00 -07:00
Hongbo Wu
dc9523e522 add searchAsYouType in elastic 2022-07-12 09:36:00 -07:00
Hongbo Wu
31ac511121 add typeahead gql schema 2022-07-12 09:36:00 -07:00
Jackson Harper
f7a023939d
Merge pull request #940 from omnivore-app/feature/allow-fetching-original 2022-07-11 22:08:27 -07:00
Hongbo Wu
c4a599d2ba support newsletters hosted on convertkit.com 2022-07-12 11:47:38 +08:00
Jackson Harper
27e39b396d Improve formatting for content display report emails 
With the previous formatting the `:` could be appended
to the end of URLs.
 2022-07-11 11:06:17 -07:00
Jackson Harper
e153966eee Linting 2022-07-09 15:38:09 -07:00
Jackson Harper
9feeb80cf3 Allow fetching original content 
This is useful when debugging
 2022-07-09 14:35:14 -07:00
Hongbo Wu
4fe81eaf92
Merge pull request #849 from omnivore-app/feature/delete-account 
Delete account API
 2022-07-09 08:54:57 +08:00
Jackson Harper
e63324c802
Merge pull request #934 from omnivore-app/fix/unsubscribe-email-not-saved 
fix/unsubscribe email not saved
 2022-07-08 13:47:16 -07:00
Jackson Harper
46d96b8f91
Merge pull request #932 from omnivore-app/newsletter-revue 
Add support for newsletters hosted on getrevue.co
 2022-07-08 13:36:10 -07:00
Hongbo Wu
de803ebbd5 fix auth error when deleting user in db 2022-07-08 11:59:55 +08:00
Hongbo Wu
ef0c2434fb Save unsubscribe email address or http url of newsletter emails 2022-07-07 17:14:12 +08:00
Hongbo Wu
322e135ead delete this user pages in elastic 2022-07-07 12:32:55 +08:00
Hongbo Wu
41f43b54ab Publish delete event 2022-07-07 12:31:46 +08:00
Hongbo Wu
efd47f3f83 add function to delete pages by param 2022-07-07 11:07:00 +08:00
Hongbo Wu
b898f55bb9 add delete account result in resolvers 2022-07-07 10:56:20 +08:00
Satindar Dhillon
e5d6386b96 return UserNotFound error if user does not exist in delete account trx 2022-07-06 09:49:01 -07:00
Satindar Dhillon
ddba50dee6 fix delete user resolver and map it to deleteAccount mutation 2022-07-06 09:38:24 -07:00
Satindar Dhillon
da76ae7aab attempt to write resolver functions 2022-07-06 09:38:24 -07:00
Satindar Dhillon
02411347a7 regenerate graphql files 2022-07-06 09:38:24 -07:00
Satindar Dhillon
d595bfb7f1 add delete account mutation to schema.ts 2022-07-06 09:38:24 -07:00
Hongbo Wu
9599edb9fb Add support for newsletters hosted on getrevue.co 2022-07-06 22:06:51 +08:00
Hongbo Wu
2e77241044 Check and get Revue newsletter url from email 2022-07-06 22:06:21 +08:00
Hongbo Wu
e5518e9f23 Reply to the original sender of the forwarded email 2022-07-06 21:16:55 +08:00
Hongbo Wu
c08afe6451 Update page updatedAt when updating labels 2022-06-28 12:03:37 +08:00
Hongbo Wu
00f38d1134 Update page updatedAt when updating highlights 2022-06-28 11:55:26 +08:00
Hongbo Wu
59efc5fea7 Add test 2022-06-28 10:14:15 +08:00
Hongbo Wu
bf1faaeea4 Include highlights in search page result in elastic 2022-06-28 09:56:39 +08:00
Hongbo Wu
6cae3dc9b5 Add highlights in search item 2022-06-28 09:53:48 +08:00
Hongbo Wu
c1855afc72 Update property unarchived=null instead of undefined in the page document in elastic 2022-06-26 18:36:54 +08:00
Jackson Harper
4c89323f0b Make the content fetch GCF URL nullable 2022-06-24 11:30:52 -07:00
Jackson Harper
92c8047a66 Update environment variable names in docker 2022-06-24 10:52:13 -07:00
Hongbo Wu
75d3372426 Use gcf for low priority task 2022-06-24 10:13:14 -07:00
Hongbo Wu
e47415cf69 Add new environment variable for content-fetch url in gcf 2022-06-24 10:13:14 -07:00
Hongbo Wu
5f0cb493f2 Allow filter search result by updatedAt 2022-06-24 13:09:13 +08:00
Hongbo Wu
e96d5552fa Fix search by is:read or is:unread getting empty result by using the correct property name 2022-06-23 12:35:48 +08:00
Hongbo Wu
9a2f4b300a Fix a script error by deleting both labels in pages and highlights 2022-06-20 19:06:50 +08:00
Hongbo Wu
f88083e535
Merge pull request #813 from omnivore-app/advanced-search 
advanced search
 2022-06-19 10:55:18 +08:00
Hongbo Wu
3f4b0f9b8c Update resolver 2022-06-17 17:09:08 +08:00
Hongbo Wu
26e7ab178e Fetch page savedAt, readingProgress from elastic 2022-06-17 17:08:26 +08:00
Hongbo Wu
2bcf97d2e8 Make updatedAt optional 2022-06-17 17:07:13 +08:00
Hongbo Wu
1be9c980e1 Add savedAt to searchItem and make readingProgress required in elastic 2022-06-17 16:03:49 +08:00
Hongbo Wu
e408143ea9 Add savedAt to searchItem and make readingProgress required 2022-06-17 16:02:46 +08:00
Hongbo Wu
e3c30dd076 Allow matching title, description, content keyword in elastic 2022-06-16 22:15:05 +08:00
Hongbo Wu
40bb49012a Allow match keyword 2022-06-16 22:15:01 +08:00
Hongbo Wu
9ec11cfc22 Add title, description, content as keyword in search.ts 2022-06-16 22:14:08 +08:00
Hongbo Wu
215078bcb3 Add logseq to the allowed cors origin 2022-06-16 10:46:06 +08:00
Hongbo Wu
c5c8763767 Add readAt to searchItem 2022-06-15 09:32:53 +08:00
Hongbo Wu
6189e85b2c Add readAt to article 2022-06-15 09:32:10 +08:00
Jackson Harper
ea4e4d6828 Use the Article type instead of Page in updatePage API 
Despite the name we are still using Article in many
places.
 2022-06-09 15:29:01 -07:00
Jackson Harper
3136705a75 Handle errors when creating a page save request on savePage 2022-06-09 13:52:49 -07:00
Jackson Harper
9baeb89742 Add more elastic debugging 2022-06-09 13:52:49 -07:00
Jackson Harper
85cbd4e9d0 Handle errors updating elastic when saving pages 2022-06-09 13:52:49 -07:00
Jackson Harper
05ea57f76f Linting 2022-06-09 12:08:42 -07:00
Jackson Harper
e23eada168 Use the public GCS URL for local uploaded files 2022-06-09 11:59:01 -07:00
Jackson Harper
ef072cb2c9 When creating pages from uploaded files use their signed URL as the page URL 2022-06-09 11:51:01 -07:00
Jackson Harper
cffb5088b4 If creating a request from a file URL, dont try fuzzy matching on URL 2022-06-09 11:37:14 -07:00
Hongbo Wu
ff613287b4
Merge pull request #775 from omnivore-app/fix/delete-label 
Fix failure of lable deletion in highlight
 2022-06-09 14:50:03 +08:00
Hongbo Wu
520ce0077b Add refresh index method 2022-06-09 12:34:22 +08:00
Hongbo Wu
4f4cc51a37 Delete label by name 2022-06-09 12:33:58 +08:00
Hongbo Wu
4cf45af669 Add debug log for deleting label 2022-06-09 12:33:42 +08:00
Hongbo Wu
85317df671
Merge pull request #774 from omnivore-app/fix/missing-title 
Fallback to domain to url if no title
 2022-06-09 11:48:49 +08:00
Hongbo Wu
d1d18315c8 Fallback to domain to url if no title 2022-06-09 11:30:13 +08:00
Jackson Harper
fa2b58eac0 When creating a page during an uploadfile request return its ID 2022-06-08 20:17:47 -07:00
Hongbo Wu
309c581cf5
Merge pull request #729 from omnivore-app/labels-in-highlights 
Support labels in highlights
 2022-06-09 09:28:43 +08:00
Hongbo Wu
2dcab66af2 Update labels for both pages and highlights 2022-06-08 17:39:38 +08:00
Hongbo Wu
7c840f5b1e Merge deleteLabel in page and highlight 2022-06-08 17:25:33 +08:00
Hongbo Wu
a981e13c86 Add test case for deleteLabel 2022-06-08 17:01:38 +08:00
Hongbo Wu
0d67d0fb90 Delete labels for highlights when label is deleted 2022-06-08 15:35:52 +08:00
Hongbo Wu
22c604c15b Add deleteLabelForHighlights 2022-06-08 15:33:43 +08:00
Hongbo Wu
145867fe05 Add test for setLabelsForHighlight resolver 2022-06-08 15:25:11 +08:00
Hongbo Wu
4894b3cfd9 Add setLabelsForHighlight in function_resolvers.ts 2022-06-08 15:00:05 +08:00
Hongbo Wu
e9f4149c62 Add setLabelsForHighlight resolver 2022-06-08 14:59:46 +08:00
Hongbo Wu
eef3ee4114 Add getLabelsByIds service method 2022-06-08 14:59:27 +08:00
Hongbo Wu
93c0e53d1d Generate setLabelsForHighlight graphql code 2022-06-08 14:59:11 +08:00
Hongbo Wu
427b58eedd Add set labels for highlight gql schema 2022-06-06 19:15:51 +08:00
Hongbo Wu
bdda54d54a Add set Labels For Highlight data access method 2022-06-06 19:15:03 +08:00
Hongbo Wu
aa15ef0282 Sort by expires at desc nulls last 2022-06-06 13:34:23 +08:00
Hongbo Wu
d00becc269 Add readAt to search API response 2022-06-04 22:12:59 +08:00
Hongbo Wu
fc68d51320 Support sort/filter by readAt 2022-06-04 22:12:03 +08:00
Hongbo Wu
b1d968ff23 Update readAt when saving reading progress 2022-06-04 22:11:44 +08:00
Hongbo Wu
577c6e84fb Add readAt to page type 2022-06-04 22:11:14 +08:00
Jackson Harper
b61a8f6015
Merge pull request #747 from omnivore-app/fix/ios-sorting 
Disable sorting options that need API improvements
 2022-06-03 14:04:19 -07:00
Jackson Harper
791e31d60f
Merge pull request #750 from omnivore-app/OMN-633 
Reopening OMN-633 to rebase
 2022-06-03 13:36:36 -07:00
Jackson Harper
b56c6dafa9 Disable sorting options that need API improvements 2022-06-03 12:47:43 -07:00
Jackson Harper
bf929db559 Add the updatedAt attribute to page items 2022-06-03 10:24:11 -07:00
Jackson Harper
975411d429
Merge pull request #719 from omnivore-app/fix/pdf-uploading-improvements 2022-06-02 16:47:03 -07:00