2536 commits

Author	SHA1	Message	Date
Jackson Harper	bdf01ddaab	Clear cache on force, return updated read position values	2024-02-01 09:59:42 +08:00
Jackson Harper	a060aaf496	Make reading position parsing safer	2024-02-01 09:59:42 +08:00
Jackson Harper	9dc070243a	Revert temp	2024-02-01 09:59:41 +08:00
Jackson Harper	d8ea4ff1c5	Add reading position messages to exported metrics for queue processor	2024-02-01 09:59:41 +08:00
Jackson Harper	7bb3718a8f	Add job to sync read position data	2024-02-01 09:59:41 +08:00
Jackson Harper	a40f5bed55	Create a service for interacting with cached read positions	2024-02-01 09:59:41 +08:00
Jackson Harper	fbfa934479	Write behind reading progress as a list that can be reduced at commit or read time	2024-02-01 09:59:39 +08:00
Jackson Harper	c2327781e1	Remove empty ctor	2024-02-01 09:57:11 +08:00
Jackson Harper	86c80d991f	Write behing cache for reading progress	2024-02-01 09:57:11 +08:00
Hongbo Wu	c3276fdaf1	Merge pull request #3466 from omnivore-app/fix/rewrite-trigger-functions-in-db ... create a job for db trigger	2024-01-31 19:47:45 +08:00
Hongbo Wu	6d5ce451ac	fix bulk action	2024-01-31 19:24:18 +08:00
Hongbo Wu	1b8111b907	wait until all jobs are done in test	2024-01-31 19:10:57 +08:00
Hongbo Wu	5e883cb2ba	running worker in the test	2024-01-31 17:45:49 +08:00
Hongbo Wu	cf898299e4	some lint	2024-01-31 14:18:34 +08:00
Hongbo Wu	5646413b59	add a job for updating highlight annotations in library_item	2024-01-31 14:15:33 +08:00
Hongbo Wu	43855247b8	update bulk action to add labels	2024-01-31 13:56:56 +08:00
Hongbo Wu	5ed35bbeff	create a job for update labels in library item	2024-01-31 13:32:37 +08:00
Hongbo Wu	ddddf8234e	create a job for db trigger	2024-01-30 23:53:05 +08:00
Hongbo Wu	36d2c21466	remove thumbnail job on complete or failure	2024-01-30 18:24:48 +08:00
Hongbo Wu	2eedba1522	fix: assign the rule job with priority 1 and do not retry	2024-01-30 18:14:34 +08:00
Hongbo Wu	e724bbd465	fallback to worker redis client if fetched content not found in cache	2024-01-30 12:27:45 +08:00
Hongbo Wu	8d8309d3bb	fix: empty client request id in save-page job data when saving pdf in rss item	2024-01-29 21:22:26 +08:00
Hongbo Wu	5faa24556e	fix: string interpolation in logs	2024-01-29 19:11:26 +08:00
Hongbo Wu	809c572ca7	reduce the size of the log entry by truncating any string values to 500 characters	2024-01-29 14:32:45 +08:00
Hongbo Wu	d5d581fc54	remove redundant logs	2024-01-29 14:32:43 +08:00
Hongbo Wu	0181a2bfcd	Merge pull request #3446 from omnivore-app/fix/refresh-feed ... fix: add rss label and feed url to the pdf in rss feed item	2024-01-29 14:21:08 +08:00
Hongbo Wu	58d384f4f6	fix pdf saving	2024-01-29 14:11:07 +08:00
Hongbo Wu	4f061546bf	remove api call to upload file request	2024-01-29 12:55:19 +08:00
Hongbo Wu	a2bdddf8c9	fix tests	2024-01-29 12:30:31 +08:00
Hongbo Wu	738f85923c	Merge pull request #3442 from omnivore-app/fix/rule-handler ... move trigger rule into a job	2024-01-29 12:20:21 +08:00
Hongbo Wu	ac5acf7f21	fix: add rss label and feed url to the pdf in rss feed item	2024-01-26 22:34:55 +08:00
Hongbo Wu	fd80724d79	fallback to guid if link of rss item is not available	2024-01-26 22:09:53 +08:00
Hongbo Wu	30fef30384	resolve conflicts	2024-01-26 20:32:56 +08:00
Hongbo Wu	e03895c8b7	enqueue trigger rule job once page is created or updated	2024-01-26 17:46:17 +08:00
Hongbo Wu	7556c6da5f	move trigger rule into a job	2024-01-26 17:45:40 +08:00
Hongbo Wu	52419c39da	resolve conflicts	2024-01-26 13:01:53 +08:00
Hongbo Wu	7e89235806	update content-fetch	2024-01-26 13:01:53 +08:00
Hongbo Wu	be0a2e8c52	use different redis server for mq and cache	2024-01-26 13:01:53 +08:00
Hongbo Wu	4c765378e3	do not retry if find_thumbnail job fails	2024-01-26 13:01:53 +08:00
Hongbo Wu	e54c691c09	create a job for finding thumbnail	2024-01-26 13:01:51 +08:00
Jackson Harper	f3c1b9d9a0	Merge pull request #3430 from omnivore-app/fix/queue-processor-custom-metrics ... Expose custom metrics for the queue processor	2024-01-25 21:17:49 +08:00
Hongbo Wu	fd7c2ffb49	fix code blocks not formatted correctly in articles from wechat official accounts	2024-01-25 16:30:59 +08:00
Hongbo Wu	dae90aaffa	fix code blocks not formatted correctly	2024-01-25 16:30:59 +08:00
Hongbo Wu	8834efabd4	throw error if save-page job failed	2024-01-25 16:30:59 +08:00
Hongbo Wu	5e239d2568	run readability in save-page instead of puppeteer	2024-01-25 16:30:59 +08:00
Jackson Harper	a639a2fa10	Add types to exported metrics	2024-01-25 12:37:02 +08:00
Jackson Harper	5f9be385b7	Remove unneeded escapes	2024-01-24 21:34:51 +08:00
Jackson Harper	c581372a93	Improve exported metrics naming convention	2024-01-24 21:13:13 +08:00
Hongbo Wu	7bb63b4ecd	for campability with old clients, use refreshedAt to replace lastFetchedAt	2024-01-24 15:32:43 +08:00
Hongbo Wu	3f3a96297f	update entity	2024-01-24 14:55:50 +08:00
Hongbo Wu	1563b44197	save the timestamp when feed fails	2024-01-24 14:54:39 +08:00
Hongbo Wu	9e55fe0d7b	do not retry if user is deleted	2024-01-24 14:12:34 +08:00
Hongbo Wu	d357d2b0af	fix tests	2024-01-24 13:41:49 +08:00
Hongbo Wu	86c4bbabee	update logs	2024-01-24 13:30:57 +08:00
Hongbo Wu	9d378d3f46	get active user subscriptions only	2024-01-24 13:21:37 +08:00
Hongbo Wu	fd74ebffa2	validate feed item url and convert relative url to absolute url	2024-01-24 13:15:46 +08:00
Hongbo Wu	6f3fccfcd0	mark subscription as error if item was failed	2024-01-24 13:09:43 +08:00
Jackson Harper	d24db92e03	Dont force init this queue	2024-01-24 12:40:00 +08:00
Jackson Harper	c29f31d618	Add _count to prom metrics names	2024-01-24 12:39:04 +08:00
Jackson Harper	e4109a63f0	Expose custom metrics for the queue processor	2024-01-24 12:15:38 +08:00
Hongbo Wu	3cb7c38e7f	update refreshed_at when feed is processed	2024-01-24 12:12:26 +08:00
Hongbo Wu	68a4fb298d	add refreshedAt to the gql schema	2024-01-24 12:03:26 +08:00
Jackson Harper	58d41951c2	Dont not fetch content for medium.com rss feeds ... All these are blocked	2024-01-24 02:11:39 +08:00
Jackson Harper	f8264bfc10	Hard code tweet url to avoid import	2024-01-23 23:45:28 +08:00
Hongbo Wu	c1d5ec722f	do not encode url when getting file data	2024-01-23 22:06:27 +08:00
Hongbo Wu	d4e487e181	set the state to failed if content is not fetched	2024-01-23 21:50:10 +08:00
Hongbo Wu	47da6fb3dc	Merge pull request #3421 from omnivore-app/fix/title ... fix: do not throw error if title is undefined in cache	2024-01-23 19:33:22 +08:00
Hongbo Wu	5495ad782e	fix: do not throw error if title is undefined in cache	2024-01-23 19:19:46 +08:00
Hongbo Wu	8f5d4ae8a0	fix: catch rejected promise if it is in a try-catch block	2024-01-23 18:45:46 +08:00
Hongbo Wu	567ce172e6	make title optional for cache in save-page job	2024-01-23 16:55:17 +08:00
Hongbo Wu	9f7669b010	fix: do not use encoded url as cache key	2024-01-22 16:49:05 +08:00
Jackson Harper	21405e6328	Fix await	2024-01-22 15:48:15 +08:00
Jackson Harper	e407db914a	Add new job for handling upload content	2024-01-22 15:36:46 +08:00
Jackson Harper	3e0f8c884e	Merge pull request #3404 from omnivore-app/debug/extra-logging ... Add logging to save_page service	2024-01-22 13:59:19 +08:00
Jackson Harper	6dd4f49c2f	Add logging to save_page service	2024-01-22 13:32:17 +08:00
Jackson Harper	4da93cbd2d	Merge pull request #3402 from omnivore-app/fix/unique-constraint ... fix: ignore error when saving existing library item in db	2024-01-22 12:38:09 +08:00
Hongbo Wu	141036dc6c	fix tests	2024-01-22 12:18:08 +08:00
Jackson Harper	db8284dc0b	Wait until backend queue is ready before putting jobs into it	2024-01-22 11:31:46 +08:00
Jackson Harper	f7b17cb93f	Only create queue object once in queue processor	2024-01-22 11:27:13 +08:00
Jackson Harper	0f78419e1a	Remove some debug	2024-01-22 11:26:59 +08:00
Hongbo Wu	5964d827c8	fix tests	2024-01-22 11:04:11 +08:00
Jackson Harper	33c994ad5c	Remove RSS feed task handler url as this handled with jobs now	2024-01-22 11:00:55 +08:00
Hongbo Wu	a807a4a18e	fix: do not throw error when creating duplicate item in db	2024-01-22 10:45:47 +08:00
Jackson Harper	9d3a183c45	Dont map label input	2024-01-21 15:44:41 +08:00
Jackson Harper	6cedffc249	Only require original content in savePageJob	2024-01-21 15:30:06 +08:00
Jackson Harper	202d19c9d3	Fix api status mapping	2024-01-21 14:11:53 +08:00
Jackson Harper	5393743c9d	Fix imports of authorized function	2024-01-21 13:33:05 +08:00
Jackson Harper	56a8ef23b6	Remove authorized function from imported helpers ... This creates some issues with init order when using the queue processor.	2024-01-21 13:22:17 +08:00
Jackson Harper	eefceeb1ef	Move checks to have the PDF check	2024-01-21 13:19:50 +08:00
Jackson Harper	9abd0a59da	Clean up imports	2024-01-21 13:16:35 +08:00
Jackson Harper	db3d7dca36	Add missing file	2024-01-21 07:43:50 +08:00
Jackson Harper	a4e707075f	Refactor savePage a bit to use it as a service from the jobs	2024-01-20 21:07:02 +08:00
Jackson Harper	2f405ebed0	Use savePage service instead of API call	2024-01-20 14:07:22 +08:00
Jackson Harper	3a18327230	Remove unused	2024-01-20 13:43:17 +08:00
Jackson Harper	7786221fbc	Fix update_subscription awaits	2024-01-20 13:38:25 +08:00
Jackson Harper	89537c13de	Simplify function input	2024-01-20 13:32:11 +08:00
Jackson Harper	1898e32607	Use service instead of API to update subscription	2024-01-20 13:27:54 +08:00
Jackson Harper	e5dec228f3	Remove some debug	2024-01-20 12:46:22 +08:00
Jackson Harper	4eeb012b78	Pass lits of fetchContentTasks into rss handlers ... This prevents the list from growing on each run	2024-01-20 11:36:14 +08:00
Jackson Harper	2b536c6086	Add more debug	2024-01-20 10:48:57 +08:00
Jackson Harper	52050728ba	Add more debugging logs	2024-01-20 10:29:07 +08:00
Jackson Harper	f4af4593c2	Fix debug line	2024-01-20 10:04:54 +08:00
Jackson Harper	4c1a182c7f	Add a refresh context to make debugging refresh runs easier	2024-01-20 09:50:21 +08:00
Jackson Harper	2961f69f02	Add a refresh context to make debugging rss jobs easier	2024-01-20 09:47:46 +08:00
Jackson Harper	a9eeda9369	Remove non-useful logging line	2024-01-20 09:23:45 +08:00
Jackson Harper	e9f22bba43	Some debug	2024-01-20 09:09:25 +08:00
Hongbo Wu	780cda6e4b	use worker redis client when connect to the queue in queue-processor	2024-01-19 20:44:54 +08:00
Hongbo Wu	338733829f	Merge pull request #3394 from omnivore-app/fix/cache-content-fetched ... fix/cache content fetched	2024-01-19 18:16:05 +08:00
Hongbo Wu	356ad38678	reduce logs	2024-01-19 18:01:09 +08:00
Hongbo Wu	4340286b89	add retry on failure in jobs	2024-01-19 17:38:25 +08:00
Jackson Harper	defb79d8c8	Use let instead of var	2024-01-19 17:12:34 +08:00
Hongbo Wu	51d5166321	get cached content from redis when saving pages	2024-01-19 17:06:37 +08:00
Jackson Harper	8ba1242ffb	Back off refresh-all-feeds if queue is very deep	2024-01-19 16:59:41 +08:00
Jackson Harper	d2491d3eb7	Reduce refresh-all-feeds priority	2024-01-19 16:59:15 +08:00
Jackson Harper	e06ff24cea	Fix logging	2024-01-19 16:44:20 +08:00
Jackson Harper	02818d94a2	Adjust priorities	2024-01-19 16:37:57 +08:00
Hongbo Wu	35805e6e8f	use user provided url for feed added	2024-01-19 16:28:37 +08:00
Jackson Harper	84375b7c36	Add validation before refreshing feed URLs	2024-01-19 16:25:05 +08:00
Jackson Harper	2dbff6a6ab	Add some more debug output	2024-01-19 16:12:44 +08:00
Jackson Harper	0e590e53a6	Improve RSS priorities for refreshing feeds	2024-01-19 16:12:22 +08:00
Hongbo Wu	d135a7fe6a	Merge pull request #3391 from omnivore-app/fix/content-fetch-dockerfile ... fix: build content-fetch in dockerfile	2024-01-19 12:45:19 +08:00
Hongbo Wu	3d42973c5f	fix api endpint	2024-01-19 12:36:15 +08:00
Jackson Harper	f03766d05b	Fix construction of service endpoint ... NEed to fix how we construct all these	2024-01-19 12:29:50 +08:00
Hongbo Wu	c2a084803a	validate feed url before adding	2024-01-18 21:56:19 +08:00
Hongbo Wu	d2c17f7539	graceful shutdown redis	2024-01-18 21:52:33 +08:00
Hongbo Wu	3e291d1019	fix tests	2024-01-18 21:13:32 +08:00
Hongbo Wu	d4f49dceb6	remove job id	2024-01-18 18:48:46 +08:00
Hongbo Wu	a6f0e2f2d9	add save page job processor	2024-01-18 18:48:46 +08:00
Hongbo Wu	8a3b50f07f	do not create thumbnail task if the item already has a thumbnail	2024-01-18 18:48:46 +08:00
Hongbo Wu	d9feb740cb	convert content-fetch to typescript	2024-01-18 18:48:46 +08:00
Jackson Harper	3f707c06f2	Fix checks for env vars	2024-01-18 16:59:53 +08:00
Jackson Harper	3b0338967d	Fix endpoint for internal refresh feed API call ... This should eventually be replaced.	2024-01-18 16:48:45 +08:00
Jackson Harper	6193c78288	Fix typo	2024-01-18 15:37:38 +08:00
Jackson Harper	7754139207	Remove debug	2024-01-18 15:33:51 +08:00
Jackson Harper	fb28d7b3d1	Enqueue new feeds at front	2024-01-18 14:34:07 +08:00
Jackson Harper	ee142d3d7c	Handle SIGINT in server better by moving DB call to after redis shutdown	2024-01-18 12:00:13 +08:00
Jackson Harper	bb55137bb4	Disconnect one client while quitting the other	2024-01-18 11:39:41 +08:00
Jackson Harper	fa685d4a29	Tear down redis with disconnect instead of quit	2024-01-18 11:28:09 +08:00
Hongbo Wu	d558fe2b17	fix tests	2024-01-18 11:24:26 +08:00
Jackson Harper	5a7056b7ec	Dont fail whole job if we cant update a subscription group	2024-01-18 10:18:45 +08:00
Jackson Harper	65347a1996	Use quit when shutting down redis	2024-01-17 17:30:16 +08:00
Jackson Harper	fd81f65e53	Remove server.close call	2024-01-17 16:37:51 +08:00
Jackson Harper	9e6b749604	Update error message	2024-01-17 16:21:42 +08:00
Jackson Harper	e1e2f86219	Remove duplicated error	2024-01-17 16:20:42 +08:00
Jackson Harper	dc4126d52c	REvert some unneeded env changes	2024-01-17 16:15:00 +08:00
Jackson Harper	361d57ac1d	Remove secrets loading as this is handled by k8s now. Fix redis init	2024-01-17 13:33:20 +08:00
Jackson Harper	d1438fe800	Remove old code, clean up redis logging	2024-01-17 11:47:31 +08:00
Jackson Harper	4f23271f29	Remove some debug	2024-01-17 11:45:05 +08:00
Jackson Harper	7a925f3fdc	Clean up redis code, remove connection close in job	2024-01-17 11:14:59 +08:00
Jackson Harper	d92ee4bcfe	Clean up redis instantiation	2024-01-17 11:03:52 +08:00
Jackson Harper	68b7b93586	More logging	2024-01-17 10:42:38 +08:00
Jackson Harper	ec765c0f5c	Some more logging	2024-01-17 10:21:19 +08:00
Jackson Harper	0936942378	Remove connect call	2024-01-16 23:32:53 +08:00
Jackson Harper	50b48a4225	Separate worker and client code for redis connections	2024-01-16 23:21:48 +08:00
Jackson Harper	c5e4e2216b	Simplify connection options	2024-01-16 23:02:46 +08:00
Jackson Harper	49734efec7	Add logging	2024-01-16 22:43:06 +08:00
Jackson Harper	7c8304ac15	Fix client imports	2024-01-16 22:08:30 +08:00
Jackson Harper	7ee3866f34	Pull in the newer connection and retry strategy code	2024-01-16 22:04:19 +08:00
Jackson Harper	90a0455c36	Update job to use ioredis	2024-01-16 22:00:25 +08:00
Jackson Harper	fcf2d07c99	Update redis data source	2024-01-16 21:58:34 +08:00
Jackson Harper	8580398d6f	Clean up post rebase	2024-01-16 21:35:46 +08:00
Jackson Harper	ba3e5b053d	Cleanup after rebase	2024-01-16 21:31:44 +08:00
Jackson Harper	f184c8baff	Fix path	2024-01-16 21:30:17 +08:00
Jackson Harper	84195f10c2	Fix endpoint	2024-01-16 21:30:17 +08:00
Jackson Harper	4df9b70c95	Fix for detecting environment	2024-01-16 21:30:17 +08:00
Jackson Harper	e050864a91	Add more debugging	2024-01-16 21:30:17 +08:00
Jackson Harper	9633e7df79	Update envar name	2024-01-16 21:30:17 +08:00
Jackson Harper	62fdb866a8	Add some extra logging	2024-01-16 21:30:17 +08:00
Jackson Harper	26d832bb92	Trim URLs from feeds	2024-01-16 21:30:17 +08:00
Jackson Harper	652f4b6180	Use createHttpTask for save operation	2024-01-16 21:30:17 +08:00
Jackson Harper	26657c5611	Use cloud task function from backend code instead of from rss-handler	2024-01-16 21:30:17 +08:00
Jackson Harper	a8e8fe23da	Remove azure config from backend	2024-01-16 21:30:17 +08:00
Jackson Harper	69eb22ae3c	Remove some env vars that are not used	2024-01-16 21:30:17 +08:00
Jackson Harper	c7f605d00a	If redis fails assume we need to refetch the item	2024-01-16 21:30:17 +08:00
Jackson Harper	00b72f1c3a	Add lwn.net to block list	2024-01-16 21:30:17 +08:00
Jackson Harper	d31d309bc0	MOre on env parsing for k8s	2024-01-16 21:30:17 +08:00
Jackson Harper	5986bd0a68	Change around redis initialization so we can pull in secrets from secret manager async on startup	2024-01-16 21:30:15 +08:00
Jackson Harper	0a82e5721e	Graceful k8s shutdown	2024-01-16 21:27:22 +08:00
Jackson Harper	7bb6602454	Fix linting	2024-01-16 21:27:22 +08:00
Jackson Harper	7aa76b70fe	Add missing jobs	2024-01-16 21:27:22 +08:00
Jackson Harper	819052f51f	Add queue-processor endpoint to the api	2024-01-16 21:27:22 +08:00
Jackson Harper	b4914f5346	Fix creation of redis clients	2024-01-16 21:27:20 +08:00
Jackson Harper	447bba23ba	Fix init	2024-01-16 21:26:30 +08:00
Jackson Harper	20df5c6212	Parse redis cert on load	2024-01-16 21:26:28 +08:00
Jackson Harper	45fcde8ebc	Update redis config	2024-01-16 21:25:34 +08:00
Jackson Harper	b4ce07efd3	Better handling of cases when redis isnt configured	2024-01-16 21:25:34 +08:00
Jackson Harper	d29c26d93b	Some redis setup	2024-01-16 21:25:34 +08:00
Jackson Harper	fc25d6d615	Enqueue rss-refresh-feed jobs in bull	2024-01-16 21:25:31 +08:00
Hongbo Wu	150a456c35	replace redis client library with ioredis	2024-01-16 15:42:50 +08:00
Hongbo Wu	1e05f7dade	log error	2024-01-16 12:17:07 +08:00
Hongbo Wu	5141c2a042	do not retry saving following items if item is saved	2024-01-16 12:15:21 +08:00
Hongbo Wu	816019e6a3	extend ttl of the saved item cache to 26 hours	2024-01-16 11:17:21 +08:00
Hongbo Wu	06067c1d66	fix tests	2024-01-15 11:26:40 +08:00
Hongbo Wu	bd612bd310	replace username too	2024-01-15 11:23:01 +08:00
Hongbo Wu	deddb51c42	fix: change user email address after user deleted account	2024-01-12 11:45:28 +08:00
Hongbo Wu	d9d800cfc4	add index for sorting by saved_at	2024-01-10 17:09:15 +08:00
Tom Rogers	451939832f	Probably a good idea to add back the schema changes...	2024-01-09 11:11:28 +00:00
Thomas Rogers	c0373646cb	Fix Linting...	2024-01-09 11:22:32 +01:00
Hongbo Wu	1ec5b7affe	fix labels not saved when merging highlights	2024-01-05 23:22:09 +08:00
Hongbo Wu	ccd4f2d3f0	catch redis client error and continue the process	2024-01-05 15:20:42 +08:00
Hongbo Wu	32a58e49f9	make redis optional for backend	2024-01-04 21:55:41 +08:00
Hongbo Wu	bcd05cc7fc	set the url in redis when an item is inserted into db	2024-01-04 18:10:14 +08:00
Hongbo Wu	08c4624fce	save the url in redis for 8 hours so rss-feeder will not try to re-save it	2024-01-04 18:03:34 +08:00
Hongbo Wu	d1db49f9bb	Merge pull request #3300 from omnivore-app/fix/batch-prune ... fix/batch prune	2024-01-04 15:19:50 +08:00
Hongbo Wu	366c97ead7	Merge pull request #3314 from omnivore-app/fix/update-event ... skip sending item update event for reading progress update	2024-01-04 15:17:08 +08:00
Hongbo Wu	d31eb7a88b	skip sending item update event for reading progress update	2024-01-04 15:06:31 +08:00
Hongbo Wu	554c4cee12	add test cases	2024-01-03 18:26:07 +08:00
Hongbo Wu	d31c520a90	add empty trash api to prune soft deleted items in trash	2024-01-03 18:26:06 +08:00
Hongbo Wu	5eb19cda71	prune users in batch	2024-01-03 18:25:16 +08:00
Jackson Harper	a1ea20b964	Update highlights query for new users	2024-01-03 18:21:13 +08:00
Jackson Harper	0b098e640f	Merge pull request #3309 from omnivore-app/fix/includes-query ... if the query contains includes:, we want to search in all folders	2024-01-03 17:48:41 +08:00
Hongbo Wu	060690609d	Merge pull request #3264 from omnivore-app/feature/prune-following-items ... add endpoint to prune items from folder older than some days	2024-01-03 17:46:21 +08:00
Hongbo Wu	6e73b10771	if the query contains includes:, we want to search in all folders	2024-01-03 17:38:12 +08:00
Hongbo Wu	b348fae677	fix multi-select bulk action api	2023-12-29 18:42:07 +08:00
Hongbo Wu	42502f9ff1	Merge pull request #3280 from omnivore-app/fix/slow-db-query ... execute update query in batch to avoid slow bulk action query	2023-12-29 16:48:18 +08:00
Hongbo Wu	7153014a87	fix duplicate labels	2023-12-29 16:36:55 +08:00
Hongbo Wu	6e32c1247d	insert labels on name conflict do nothing	2023-12-29 16:11:11 +08:00
Hongbo Wu	943db3332e	Merge pull request #3276 from omnivore-app/fix/in-library ... by default we want to search in the inbox if no in: is specified	2023-12-29 14:51:00 +08:00
Hongbo Wu	cbb07318e5	get the unchanged result set of sub query by checking updated_at	2023-12-29 12:53:49 +08:00
Hongbo Wu	b5d84098d0	increase batch size to 100	2023-12-29 09:53:23 +08:00
Hongbo Wu	de027ee93f	use cursor and fetch instead of limit and offset because the subquery result set could change during update	2023-12-29 09:53:23 +08:00
Hongbo Wu	55518139fb	generate raw sql because postgres does not support prepared statements in DO blocks	2023-12-29 09:53:23 +08:00
Hongbo Wu	1079a19cd9	execute update query in batch to avoid slow bulk action query	2023-12-29 09:53:23 +08:00
Hongbo Wu	881626d35a	fix type of highlight is null from update since api	2023-12-28 16:55:23 +08:00
Hongbo Wu	2b886d7e68	return highlights in update since api response	2023-12-28 14:03:07 +08:00
Hongbo Wu	47aa2eaae5	by default we want to search in the inbox if no in: is specified	2023-12-22 18:01:38 +08:00
Hongbo Wu	c21ba9e11f	fix tests	2023-12-22 17:12:24 +08:00
Hongbo Wu	ccec94dae0	insert labels only if name unique	2023-12-22 15:14:07 +08:00
Hongbo Wu	c939735678	remove invalid-url error from subscribing feed	2023-12-22 09:51:27 +08:00
Hongbo Wu	f6e04daf60	return invalid url if feed url is different from user provided url	2023-12-22 09:05:43 +08:00
Hongbo Wu	696de8ea50	check both user provided url and parsed feed url when looking for existing subscription	2023-12-22 08:08:15 +08:00
Hongbo Wu	db28bbd0f9	fix invalid unknown item type	2023-12-20 11:51:19 +08:00
Hongbo Wu	0a4758105a	feat: update newsletter email api	2023-12-20 11:29:10 +08:00
Hongbo Wu	e4b6cba2ff	Merge pull request #3261 from omnivore-app/feature/add-state-for-non-fetch-content ... feature/add state for non fetch content	2023-12-19 18:30:13 +08:00
Hongbo Wu	92aa11b98e	add fetchContent api	2023-12-19 18:05:02 +08:00
Hongbo Wu	07b96c46d5	fix tests	2023-12-19 17:45:12 +08:00
Hongbo Wu	b5f82d448e	return folder when newsletter email api is called	2023-12-19 14:52:30 +08:00
Hongbo Wu	8e77c047bd	add endpoint to prune items from folder older than some days	2023-12-19 14:25:55 +08:00
Hongbo Wu	22778af285	run readability on the preview content	2023-12-19 10:41:20 +08:00
Hongbo Wu	b1aca117f4	if fetch_content is false, set the state to content_not_fetched and use preview content	2023-12-19 10:41:20 +08:00
Hongbo Wu	6f1ef075c6	add content_not_fetched state for items saved without content fetched	2023-12-19 10:41:20 +08:00
Hongbo Wu	15dc05ca1a	fix sending rss feed to inbox folder	2023-12-18 19:52:44 +08:00
Hongbo Wu	55ed5a0a80	fix typo	2023-12-18 16:20:40 +08:00
Hongbo Wu	97735d0633	make existing newsletters go to inbox folder	2023-12-18 16:08:36 +08:00
Hongbo Wu	7a2cf6c6fe	fix sort by word count not working	2023-12-18 10:40:10 +08:00
Hongbo Wu	58f16ec62c	Merge pull request #3247 from omnivore-app/feature/feeds ... feat: fetch content and folder settings for newsletters and subscriptions	2023-12-15 16:46:02 +08:00
Hongbo Wu	d579a8f907	not throw error if sort keyword not found	2023-12-15 13:31:22 +08:00
Hongbo Wu	f915c18dfa	allow plural and singular form for has/no filter	2023-12-15 13:18:25 +08:00
Hongbo Wu	42749c5a32	throw error if sort column not exists	2023-12-15 11:47:00 +08:00
Hongbo Wu	b86ee139e0	move to folder api will create a task to fetch content if content not exists	2023-12-14 19:20:03 +08:00
Hongbo Wu	83e0982457	save newsletters to the right folder get from subscription and email settings	2023-12-14 18:53:32 +08:00
Hongbo Wu	f7ec0220a8	allow setting folder when creating a new email	2023-12-14 17:48:02 +08:00
Hongbo Wu	32f99d7b07	allow updating folder and fetch content settings for subscriptions	2023-12-14 17:27:38 +08:00
Hongbo Wu	a257147fc5	fetch following item content only when required	2023-12-14 17:25:55 +08:00
Hongbo Wu	f2c468f906	allow setting fetchContent and folder when subscribe to a feed	2023-12-14 16:32:48 +08:00
Hongbo Wu	d67b974367	add fetch_content and folder column to the subscriptions table and folder column to newsletter_emails table	2023-12-14 15:41:54 +08:00
Hongbo Wu	06e17e2eb8	return source of the labels tagged to an item	2023-12-14 10:40:18 +08:00
Hongbo Wu	dcd6e8bb60	allow adding source of label when adding labels	2023-12-13 16:25:25 +08:00
Hongbo Wu	56a7ae4dc1	add source column to entity_labels table	2023-12-13 14:35:24 +08:00
Hongbo Wu	79baee56c6	Merge pull request #3242 from omnivore-app/fix/subscribe-api ... fix/subscribe api	2023-12-12 20:01:40 +08:00
Hongbo Wu	528163905d	Merge pull request #3244 from omnivore-app/improve-advanced-search ... improve advanced search	2023-12-12 19:53:29 +08:00
Hongbo Wu	9b8e4a6876	fix lint	2023-12-12 19:48:08 +08:00
Hongbo Wu	2155be4704	fix tests	2023-12-12 19:42:09 +08:00
Hongbo Wu	07868c729d	update comments	2023-12-12 18:13:38 +08:00
Hongbo Wu	a7f9214cda	ignore empty text field	2023-12-12 17:56:10 +08:00
Hongbo Wu	609c46cdc3	ignore empty field value	2023-12-12 16:35:50 +08:00
Hongbo Wu	7622d8a139	ignore field case	2023-12-12 16:17:31 +08:00
Hongbo Wu	55cc5f5769	treat unknown fields as text search	2023-12-12 16:14:51 +08:00
Hongbo Wu	a41ec3d2f6	lint	2023-12-12 15:38:39 +08:00
Hongbo Wu	e730abb9aa	fix subscribe api returns error due to createdAt being null	2023-12-12 15:17:00 +08:00
Hongbo Wu	1056971e89	copy published date to the recommended item	2023-12-12 11:17:12 +08:00
Jackson Harper	a7c74cef6f	Merge pull request #3236 from omnivore-app/scan-feed-api ... update scan feed api for the client	2023-12-11 15:35:51 +08:00
Hongbo Wu	b3d0bb9ed7	return absolute feed url	2023-12-11 14:14:54 +08:00
Hongbo Wu	9f59d09c20	validate and return rss feed if the url is an rss feed url	2023-12-11 13:46:35 +08:00
Hongbo Wu	36896a8770	max rss feed defaults to 256	2023-12-11 13:18:43 +08:00
Hongbo Wu	b135dfbc46	get max feed limit from env	2023-12-11 13:01:56 +08:00
Hongbo Wu	4d41117d8b	ignore sort:score since we always order by score when doing fulltext search	2023-12-11 11:54:56 +08:00
Hongbo Wu	0d60ece6d6	make folder optional	2023-12-08 20:03:36 +08:00
Hongbo Wu	da3fede3a4	fix saved search bug: undefined category in input	2023-12-08 19:58:41 +08:00
Hongbo Wu	5e566d450e	update bulk action api	2023-12-08 16:50:39 +08:00
Hongbo Wu	1697f11e75	wrap the where query in parenthesis because we also need to check user_id	2023-12-08 16:49:07 +08:00
Hongbo Wu	36777568ac	create/add RSS labels to the library item when feedUrl is there	2023-12-08 11:29:03 +08:00
Hongbo Wu	3759e10615	fix feed url in pdf file not saved	2023-12-08 11:29:02 +08:00
Hongbo Wu	07d8c72972	create a standalone version of liqe	2023-12-07 18:47:25 +08:00
Hongbo Wu	bf8d081161	wrap non-english words in quotes	2023-12-07 17:59:45 +08:00
Hongbo Wu	5f09165128	allow bulk action on max 100 items	2023-12-07 10:55:25 +08:00
Hongbo Wu	ed29721728	fix bulk action query limited to 1000 char	2023-12-06 22:06:47 +08:00
Hongbo Wu	d42f257f9e	fix searching for multiple labels in iOS	2023-12-06 21:09:30 +08:00
Hongbo Wu	dff2376df3	fix integration handler to use first as number	2023-12-06 14:44:07 +08:00
Hongbo Wu	4ec16a1752	fix parenthesized expression	2023-12-06 14:02:23 +08:00
Hongbo Wu	284697da5b	fix sort	2023-12-06 13:52:17 +08:00
Hongbo Wu	c8bd953cf3	capture the right value of the rss feed url in search string	2023-12-06 10:46:15 +08:00
Hongbo Wu	8579b79eb3	fix tests	2023-12-05 20:58:08 +08:00
Hongbo Wu	4dacceac01	allow use keyword	2023-12-05 20:43:23 +08:00
Hongbo Wu	29e2bf45e3	allow use:folders in search string for new clients to filter inbox items	2023-12-05 20:40:02 +08:00
Hongbo Wu	7c5f90983e	clean up	2023-12-05 13:05:31 +08:00
Hongbo Wu	e6322ffaf8	remove unused code	2023-12-05 13:04:07 +08:00
Hongbo Wu	3c67d1911c	fix tests	2023-12-05 12:31:26 +08:00
Hongbo Wu	073bd8b43a	fix in:trash	2023-12-05 10:58:20 +08:00
Hongbo Wu	88a61d962f	parse sort parameter	2023-12-05 10:58:20 +08:00
Hongbo Wu	a3d3f4daf4	feat: allow logical operator in search query	2023-12-05 10:58:18 +08:00
Hongbo Wu	120fe19dab	fix typo	2023-12-04 18:53:46 +08:00
Hongbo Wu	a116833659	feat: allow moving items to a folder in a bulk	2023-12-04 18:10:14 +08:00
Hongbo Wu	37beab64c6	Merge pull request #3198 from omnivore-app/fix/move-to-folder-api ... fix moveToFolder API error by skipping fetching content again	2023-12-04 16:10:09 +08:00
Hongbo Wu	767629b8c7	replace articleSavingRequest with success (boolean value) as a response	2023-12-04 15:54:14 +08:00
Hongbo Wu	824b8187af	use system token in integration handler	2023-12-04 15:38:08 +08:00
Hongbo Wu	bafcdb2fee	skip rate limiter for system requests	2023-12-04 14:56:26 +08:00
Hongbo Wu	e33c4ad685	fix moveToFolder API error by skipping fetching content again	2023-12-04 14:29:54 +08:00
Hongbo Wu	fe86085245	update comment	2023-12-01 12:19:05 +08:00
Hongbo Wu	706ff94cd1	allow use:folders in search string for new clients to filter inbox items	2023-12-01 12:11:09 +08:00
Hongbo Wu	bd3945fc7e	Merge pull request #3164 from omnivore-app/feature/fetch-content-for-following ... feat: fetch content for rss feed items in following folder	2023-11-28 12:07:17 +08:00
Hongbo Wu	af5040a518	saving posts from telegram channel as rss items	2023-11-24 17:23:35 +08:00
Hongbo Wu	8075cfd431	allow adding telegram channel as feed	2023-11-24 16:06:24 +08:00
Hongbo Wu	989f4f04e0	add scan feeds api to scan feeds from opml or html	2023-11-24 12:11:33 +08:00
Hongbo Wu	022d0f9e66	Merge pull request #3159 from omnivore-app/fix/nested-labels ... feat: wildcard search for labels and nested labels	2023-11-23 21:44:44 +08:00
Hongbo Wu	fd781644f1	feat: fetch content for rss feed items in following folder	2023-11-23 18:03:25 +08:00
Hongbo Wu	750b59cb8d	Merge pull request #3139 from omnivore-app/change-folder-to-state-1	2023-11-23 12:31:37 +08:00
Hongbo Wu	46f3515698	feat: wildcard search for labels and nested labels	2023-11-22 23:46:55 +08:00
Hongbo Wu	4e60396d17	save thumbnail of the following item	2023-11-20 17:28:34 +08:00
Hongbo Wu	e26ed0b6f0	add RSS label to the following feed items	2023-11-17 18:26:00 +08:00
Hongbo Wu	c5a97c7a41	change archive and trash from folder to state	2023-11-17 13:51:33 +08:00
Hongbo Wu	d5d3c72196	Merge pull request #3135 from omnivore-app/change-folder-to-state ... Change category to folder in saved search	2023-11-17 12:31:21 +08:00
Hongbo Wu	fca406bb3d	allow updating taskName	2023-11-17 11:27:30 +08:00
Hongbo Wu	24f31ce1eb	add folder to the updates api params	2023-11-17 11:27:30 +08:00
Hongbo Wu	0869cfd477	replace category with folder in the GQL type	2023-11-16 17:50:30 +08:00
Hongbo Wu	d82eda1f32	allow importing all or unarchived only pocket items	2023-11-16 16:37:14 +08:00
Hongbo Wu	25d0123007	fix tests	2023-11-15 17:41:28 +08:00
Hongbo Wu	f9b389edad	add folder to article and search item type	2023-11-15 17:25:14 +08:00
Hongbo Wu	12c18a3661	add preview content to the search API response	2023-11-15 15:53:01 +08:00
Hongbo Wu	8b92e96db1	generate slug	2023-11-15 14:27:54 +08:00
Hongbo Wu	c6807a554a	item_id and saved_at are optional in the post request	2023-11-15 12:34:35 +08:00
Hongbo Wu	c59b4ca97e	Merge pull request #3054 from omnivore-app/feature/social-feeds ... feature: following	2023-11-15 11:19:14 +08:00
Hongbo Wu	609d30adbb	compatiblity with old saved searches	2023-11-15 10:22:58 +08:00
Hongbo Wu	3d84108202	remove feeds table	2023-11-14 15:27:58 +08:00
Hongbo Wu	2ade0698c5	remove unused code	2023-11-10 23:03:01 +08:00
Hongbo Wu	59bbc5562b	fix tests	2023-11-10 22:37:14 +08:00
Hongbo Wu	adf13131db	add folder to library_item table	2023-11-10 17:34:46 +08:00
Hongbo Wu	69dc1ba47a	fix tests	2023-11-09 22:39:51 +08:00
Hongbo Wu	98cc0e2b82	upsert feed table when subscribing	2023-11-09 22:06:30 +08:00
Hongbo Wu	490b40e467	fix tests	2023-11-09 19:08:52 +08:00
Hongbo Wu	9dfc5f856d	rename variables	2023-11-09 19:00:57 +08:00
Hongbo Wu	ffac5c195d	update comment	2023-11-09 16:13:31 +08:00
Hongbo Wu	f96ba17c92	fetch rss content if required	2023-11-09 16:09:23 +08:00
Hongbo Wu	b7496db56c	add following handler to save following item	2023-11-09 15:53:44 +08:00
Hongbo Wu	cbb5a0a921	revert searchItem changes	2023-11-08 18:16:48 +08:00
Hongbo Wu	cc53992b9a	add addFollowingToLibrary API	2023-11-08 18:16:48 +08:00
Hongbo Wu	99616501cf	add saveFollowing API	2023-11-08 18:16:48 +08:00
Hongbo Wu	3ff9ce3a09	add in:following search filter	2023-11-08 18:16:48 +08:00
Hongbo Wu	98faefa644	add date variables to date search	2023-11-08 18:16:48 +08:00
Hongbo Wu	2c9ec454f5	fix tests	2023-11-08 18:16:48 +08:00
Hongbo Wu	9b0b378c39	check is_fetching_content and only fetch content if required for rss feed items	2023-11-08 18:16:48 +08:00
Hongbo Wu	679512c9d1	add columns to library_item	2023-11-08 18:16:48 +08:00
Hongbo Wu	1da8dc6545	update library_item table	2023-11-08 18:16:48 +08:00
Hongbo Wu	da0d1705b5	add following graphql api to fetch user following	2023-11-08 18:16:48 +08:00
Hongbo Wu	fc783f01ec	add feeds graphql api for searching feed by title or url	2023-11-08 18:16:48 +08:00
Hongbo Wu	878d881e05	add feed entity	2023-11-08 18:16:48 +08:00
Hongbo Wu	e282eea99f	add feeds graphql api to the schema	2023-11-08 18:16:48 +08:00
Jackson Harper	1e336dd61c	Use CLIENT_URL instead of hard coded values	2023-11-08 18:16:32 +08:00
Hongbo Wu	3e9a049306	add fields to the response of user personalization api	2023-11-07 15:39:56 +08:00
Hongbo Wu	fdad3a28af	Merge pull request #3024 from omnivore-app/integratoin ... improvement on integrations	2023-11-03 17:20:17 +08:00
Hongbo Wu	3761b396f4	remove unused code	2023-11-03 10:17:36 +08:00
Hongbo Wu	9ef608b4b6	fix: highlights not added to the content if searching for highlighted content	2023-11-03 10:11:14 +08:00
Hongbo Wu	ffad064d3a	Merge pull request #3051 from naveennamani/patch-1 ... Fixes #3050 - URLs sensitive to trailing slash	2023-11-01 15:48:04 +08:00
Hongbo Wu	e98d029250	remove unused fields in library_item entity so we exclude them from the webhook event	2023-11-01 14:56:36 +08:00
Hongbo Wu	c93fc14de8	remove user from highlight webhook event	2023-11-01 13:26:36 +08:00
Hongbo Wu	30cc952f02	do not publish events if the highlights are recommended by club users	2023-11-01 13:24:06 +08:00
Naveen Namani	ef1adfee98	Fixes #3050 - URLs sensitive to trailing slash ... By passing false to `removeTrailingSlash` for normalize-url call, the URLs sensitive to trailing slash will be parsed correctly	2023-10-30 14:31:46 +05:30
Hongbo Wu	8df6fb3ae2	import unarchived items by default	2023-10-30 16:22:09 +08:00
Hongbo Wu	f6c0b2ba34	add import_item_state to set integration api payload	2023-10-30 16:09:10 +08:00
Hongbo Wu	a528692125	add import_item_state to integration entity	2023-10-30 15:12:18 +08:00
Hongbo Wu	07eb97e7cc	remove unused code	2023-10-30 14:41:53 +08:00
Hongbo Wu	58d5eb6bc0	create an API to create a cloud task for each active integration when the cronjob triggers	2023-10-30 14:41:52 +08:00
Hongbo Wu	5edba30e23	create an integration handler for syncing with readwise	2023-10-30 14:41:52 +08:00
Hongbo Wu	47f67c237d	change source to pocket if item is imported from pocket	2023-10-30 14:41:52 +08:00
Hongbo Wu	6bdc7f01e1	Merge pull request #3027 from omnivore-app/fix/site-search ... fix: site scoped search not working for domain and hostname	2023-10-30 11:08:26 +08:00
Jackson Harper	6db23ba05c	Add intercom identity verification	2023-10-27 16:13:03 +08:00
Hongbo Wu	d480d53151	fix: site scoped search not working for domain and hostname ... * alter site_name_tsv to include hostname and domain * add test cases	2023-10-26 22:10:35 +08:00
Hongbo Wu	a6e8362401	add debugging logs	2023-10-26 12:56:46 +08:00
Hongbo Wu	cf95b6fa07	rename the endpoint to /prune	2023-10-25 14:41:49 +08:00
Hongbo Wu	dddb51f7d5	add test cases	2023-10-25 13:14:49 +08:00
Hongbo Wu	5f5423ddcb	add a pubsub service endpoint for cleanup soft deleted users	2023-10-25 13:02:59 +08:00
Hongbo Wu	c2092e0e5d	Merge pull request #2995 from omnivore-app/fix/item-id-update ... fix: library item id could be updated if a different client request id supplied in save page api payload	2023-10-24 15:24:18 +08:00
Jackson Harper	533e3554fd	Merge pull request #2997 from omnivore-app/feature/reading-progress-api ... ignore position check if force = true in reading progress api payload	2023-10-24 14:45:47 +08:00
Hongbo Wu	056bf9ebf1	fix update sql	2023-10-24 13:12:59 +08:00
Hongbo Wu	3c75a23e75	fix: showing original url instead of signed url for pdf files in search api response	2023-10-24 12:02:58 +08:00
Hongbo Wu	41866d5cb7	cont	2023-10-23 22:02:43 +08:00
Hongbo Wu	3f14ac4f9b	skip publishing event for importing	2023-10-23 21:31:38 +08:00
Hongbo Wu	79a647deb4	do not publish content and original html in the queue	2023-10-23 21:27:16 +08:00
Hongbo Wu	f46bb616fa	update read_at to now	2023-10-23 21:14:23 +08:00
Hongbo Wu	e673db423b	limit max content size to 10MB	2023-10-23 21:11:37 +08:00
Hongbo Wu	f18ea7a7ce	ignore position check if force = true in reading progress api payload	2023-10-23 20:57:20 +08:00
Hongbo Wu	0c60706503	fix: date filter in bulk action api	2023-10-23 19:34:41 +08:00
Hongbo Wu	27b352f9ae	fix: updates since api returns error for android client if invalid date is supplied	2023-10-23 19:07:47 +08:00
Hongbo Wu	e94a61a9bc	fix: library item id could be updated if a different client request id supplied in save page api payload	2023-10-23 17:44:38 +08:00
Hongbo Wu	b26e15c5d6	lint	2023-10-23 15:44:47 +08:00
Hongbo Wu	bbc46c6389	fix: when source is rss-feed or csv-import, do not parse items in the backend	2023-10-23 15:43:19 +08:00
Hongbo Wu	850569d7e4	add test cases	2023-10-23 12:24:19 +08:00
Hongbo Wu	7763a8a939	perf: limit search item results to 100 a time and fetch labels and recommendations only needed	2023-10-23 10:52:03 +08:00
Jackson Harper	1ebfb94309	Merge pull request #2977 from Rudra-Ji/main ... fix typo	2023-10-20 18:00:43 +08:00
Hongbo Wu	b2a61b2d7c	Merge pull request #2969 from omnivore-app/fix/query-group ... Group RSS subscriptions by feed URL and filter by scheduled time	2023-10-20 14:45:44 +08:00
Hongbo Wu	6618a6ed19	migrate popular reads into database	2023-10-20 14:31:16 +08:00
Rudra	23eb7ea76f	fix typo	2023-10-20 11:51:02 +05:30
Jackson Harper	a23b67b70b	Dont rate limit auth in local env so tests can run	2023-10-20 11:44:52 +08:00
Jackson Harper	b9e2f9ee53	Add a longer rate limit window on createaccount/reset password, reduce api rate limit hits	2023-10-20 11:28:16 +08:00
Hongbo Wu	4e36a5809e	fix failure of re-subscribing rss feed	2023-10-20 11:05:58 +08:00
Hongbo Wu	750ccb0c42	fix conflicts with main	2023-10-19 17:44:53 +08:00
Hongbo Wu	1880f2ace4	Update rss handler to fetch a group of rss feeds in one task	2023-10-19 17:27:55 +08:00
Hongbo Wu	f50d1c9cb9	Group RSS subscriptions by feed URL and create cloud task for the groups	2023-10-19 17:27:08 +08:00
Hongbo Wu	5283a84908	add scheduledAt to the task payload	2023-10-19 17:27:08 +08:00
Hongbo Wu	18aece84e7	add scheduledAt to subscriptions	2023-10-19 17:27:08 +08:00
Jackson Harper	c93e61ec24	Remove debug	2023-10-18 21:30:23 +08:00
Jackson Harper	78a642db6d	Update type signature	2023-10-18 19:53:26 +08:00
Jackson Harper	5c576347a8	MOre debug	2023-10-18 19:26:45 +08:00
Jackson Harper	fbaaaeca7c	Add some debugging	2023-10-18 18:48:18 +08:00
Jackson Harper	7848413645	Update schema	2023-10-18 16:34:20 +08:00
Jackson Harper	cf101c6d18	Cache and check feed checksums to reduce fetching	2023-10-18 16:26:03 +08:00
Hongbo Wu	46313b14ba	set trust proxy = true if set in env var	2023-10-18 13:06:33 +08:00
Hongbo Wu	988b20e30c	test trust proxy=true	2023-10-18 12:30:27 +08:00
Hongbo Wu	506004d410	test trust proxy=2	2023-10-18 11:58:15 +08:00
Hongbo Wu	32ba744d20	test trust proxy=1	2023-10-18 11:04:32 +08:00
Hongbo Wu	91076014cd	log req.ip	2023-10-18 11:04:32 +08:00
Hongbo Wu	0874f6ff21	log x-forwarded-for header for debugging purpose	2023-10-18 11:04:32 +08:00
Jackson Harper	71aaef3628	Update attachment URL	2023-10-18 10:43:53 +08:00
Jackson Harper	b88c64424f	Special handling of file URLs	2023-10-18 10:37:21 +08:00
Jackson Harper	78843fff14	Remove import	2023-10-18 10:27:09 +08:00
Jackson Harper	4220922ff8	Preserve original URL if possible	2023-10-18 10:23:26 +08:00
Jackson Harper	b6ce4f39b1	Fix imports	2023-10-18 10:15:49 +08:00
Jackson Harper	25c0051fd4	Only use signed URLs for PDF attachments	2023-10-18 10:14:35 +08:00
Hongbo Wu	fa2fba1a2c	Merge pull request #2942 from omnivore-app/fix/rules ... improvement on subscriptions and delete user api	2023-10-17 17:14:49 +08:00
Hongbo Wu	700fd73dbb	disallow social login if account is deleted	2023-10-17 17:07:46 +08:00
Hongbo Wu	6e79f6c67f	update sql to deduplicate rss subscriptions	2023-10-17 16:41:14 +08:00
Hongbo Wu	eb667e04b9	update reading progress in db	2023-10-17 15:09:16 +08:00
Hongbo Wu	1b2d93e118	remove duplicate rss subscriptions by user and url; create a unique constraint	2023-10-16 17:35:29 +08:00
Jackson Harper	9e8ccb35e3	Limit users to a single import a day	2023-10-16 15:48:03 +08:00
Hongbo Wu	e54c1c81a1	soft delete user by calling delete account api	2023-10-16 15:24:53 +08:00
Jackson Harper	34798f39f0	Disable JSONLD fetching which can be quite slow	2023-10-16 11:25:37 +08:00
Jackson Harper	a20b956e26	Merge pull request #2926 from omnivore-app/fix/api-site-name-query ... Add ILIKE queries to all field queries	2023-10-13 22:02:55 +08:00
Hongbo Wu	497b6e9f72	fetch highlights if necessary and before converting content to markdown	2023-10-13 18:27:57 +08:00
Jackson Harper	3f4be32df8	Merge library item imports	2023-10-13 18:26:02 +08:00
Hongbo Wu	a6c5f4e862	fix bulk action slow query	2023-10-13 13:45:09 +08:00
Hongbo Wu	4d64231abf	save content reader in the item	2023-10-12 12:59:36 +08:00
Hongbo Wu	d1472c6519	Merge pull request #2915 from omnivore-app/fix/drop-label-position-trigger ... drop position trigger on labels table and sort labels by name returned by labels api	2023-10-12 12:10:33 +08:00
Hongbo Wu	9b25a64fd8	Merge pull request #2916 from omnivore-app/fix/email-attachment ... fix email attachment not being saved as item	2023-10-12 12:09:01 +08:00
Hongbo Wu	039632556b	add more method in mock cloud storage and update test cases for email-attachment rest api	2023-10-12 11:56:54 +08:00
Hongbo Wu	1bc9912271	fix failure of saving highlight when position is null	2023-10-12 11:27:46 +08:00
Hongbo Wu	ebb45f1973	fix email attachment not being saved as item	2023-10-12 10:47:20 +08:00
Hongbo Wu	f1c4bc6a74	fix email attachment not being saved as item	2023-10-12 10:46:12 +08:00
Hongbo Wu	b0f0dba53d	drop position trigger on labels table and sort labels by name returned by labels api	2023-10-12 10:39:35 +08:00
Hongbo Wu	30ba22cbbc	add user_id to the sql query to reduce query latency	2023-10-11 16:05:35 +08:00
Hongbo Wu	a4aeb40276	fix: after successfully deleted highlights, still return error codes	2023-10-11 14:48:44 +08:00
Hongbo Wu	768845e101	update the filter value to use is:reading of the Continue Reading filter	2023-10-10 15:33:59 +08:00
Hongbo Wu	62ffaa924b	add wordsCount filter	2023-10-10 12:52:49 +08:00
Hongbo Wu	93c410a564	fix is:read filter and add is:reading filter	2023-10-10 12:21:32 +08:00
Hongbo Wu	3e91f3836b	fix tests	2023-10-10 10:58:48 +08:00
Hongbo Wu	835b61064f	allow filter by wordsCount	2023-10-10 10:58:48 +08:00
Hongbo Wu	1adbed9876	revert postgres connection timeout	2023-10-10 10:33:49 +08:00
Hongbo Wu	bab343c41f	testing	2023-10-10 10:10:45 +08:00
Hongbo Wu	ac0d295cf3	debug	2023-10-09 22:06:35 +08:00
Hongbo Wu	0b7da62aca	debug	2023-10-09 21:34:52 +08:00
Hongbo Wu	f4f38e4b31	reduce unnecessary logs	2023-10-09 17:26:03 +08:00
Hongbo Wu	78490e15f1	disable db connection timeout	2023-10-09 15:30:30 +08:00
Hongbo Wu	50901d25a3	remove elastic url from env var	2023-10-09 13:00:30 +08:00
Hongbo Wu	e424328f20	fix recommendations not fetched in search results	2023-10-05 23:00:29 +08:00
Hongbo Wu	ac576aa5c7	fix isArchived = false when archivedAt is not null	2023-10-05 17:53:57 +08:00
Hongbo Wu	39b3a1cad4	resolve conflicts	2023-10-05 14:37:21 +08:00
Hongbo Wu	463231b26b	fix trigger	2023-10-05 14:32:05 +08:00
Hongbo Wu	0e7f45a758	create a trigger to update recommender_names in library_item table	2023-10-05 14:32:05 +08:00
Hongbo Wu	f6c906201b	fetch recommendations separately too	2023-10-05 14:32:05 +08:00
Hongbo Wu	d4404b0248	fetch labels and highlights after items fetched in a search query	2023-10-05 14:32:05 +08:00
Hongbo Wu	bb59ed757c	fix word count not return	2023-10-05 14:32:05 +08:00
Hongbo Wu	cbf58614c4	use log levels in custom typeorm logger	2023-10-05 14:32:05 +08:00
Hongbo Wu	efdbc4f345	use interpolated string or dictionary parameter in logs	2023-10-05 14:32:03 +08:00
Hongbo Wu	327f6959ef	fix highlight createdByMe is wrong	2023-10-05 14:31:07 +08:00
Hongbo Wu	0ff9fdef04	add elastic scroll time in the env var	2023-10-05 14:31:07 +08:00
Hongbo Wu	0912e7cf0f	remove warn logs	2023-10-05 14:31:07 +08:00
Hongbo Wu	3b8b48bc21	fix recovered deleted items still shown in trash	2023-10-05 14:31:07 +08:00
Hongbo Wu	7bf3f1bbf4	set deletedAt to now when bulk delete	2023-10-05 14:31:07 +08:00
Hongbo Wu	47510c0785	fix recommendations not fetched in search results	2023-10-05 14:31:07 +08:00
Hongbo Wu	9c723ecbd3	fix id not set in page update pubsub message	2023-10-05 14:31:07 +08:00
Hongbo Wu	0c691257eb	fix rules permission	2023-10-05 14:31:07 +08:00
Hongbo Wu	5f6587537e	fix recommendation wildcard search	2023-10-05 14:31:07 +08:00
Hongbo Wu	bf2c54084b	fix getArticle API failed on iOS	2023-10-05 14:31:07 +08:00
Hongbo Wu	9e5196406e	revert rls on old tables	2023-10-05 14:31:07 +08:00
Hongbo Wu	2e5529fd81	allow readingProgressAnchorIndex to be null	2023-10-05 14:31:07 +08:00
Hongbo Wu	8712107a2f	remove db caching	2023-10-05 14:31:07 +08:00
Hongbo Wu	c57ead3583	add test for when rss:feed in:archive is in the query	2023-10-05 14:31:07 +08:00
Hongbo Wu	867c31a582	add test for searching in:archive label:test	2023-10-05 14:31:07 +08:00
Hongbo Wu	6cb4bb8de8	fix in:subscription search	2023-10-05 14:31:07 +08:00
Hongbo Wu	15f9017291	fix in:archive search	2023-10-05 14:31:07 +08:00
Hongbo Wu	a35e6c0147	fix sort by read time bug	2023-10-05 14:31:07 +08:00
Hongbo Wu	dc98719b85	reduce log	2023-10-05 14:31:07 +08:00
Hongbo Wu	0c5ba53c26	fix recommendations	2023-10-05 14:31:07 +08:00
Hongbo Wu	b9af543b45	fix rss feeds	2023-10-05 14:31:06 +08:00
Hongbo Wu	673ce83c33	fix searching for site	2023-10-05 14:31:06 +08:00
Hongbo Wu	4ff5484d8e	change item_type to text	2023-10-05 14:31:06 +08:00
Hongbo Wu	7c56973fad	fix update filter api	2023-10-05 14:31:06 +08:00
Hongbo Wu	e696709a69	resolve conflicts	2023-10-05 14:31:06 +08:00
Hongbo Wu	7225d861f3	use skip and take instead of limit, offset	2023-10-05 14:31:06 +08:00
Hongbo Wu	68e381deb2	revert note changes	2023-10-05 14:31:06 +08:00
Hongbo Wu	fdd771bc06	add migration script	2023-10-05 14:31:06 +08:00
Hongbo Wu	dd292879a4	sort type ahead search result by saved at desc order	2023-10-05 14:31:06 +08:00
Hongbo Wu	2b9a6b26f6	debug test error	2023-10-05 14:30:37 +08:00
Hongbo Wu	a284884398	fix rebase conflicts	2023-10-05 14:30:37 +08:00
Hongbo Wu	5a9dc344d3	fix rebase conflicts	2023-10-05 14:30:35 +08:00
Hongbo Wu	cbaa3d6900	fix newsletters test	2023-10-05 14:30:10 +08:00
Hongbo Wu	c3caf15e44	fix auth tests	2023-10-05 14:30:10 +08:00
Hongbo Wu	024d558cfe	update labels rls	2023-10-05 14:30:10 +08:00
Hongbo Wu	6672ca0109	fix labels tests	2023-10-05 14:30:10 +08:00
Hongbo Wu	4b95b732cf	fix highlight tests	2023-10-05 14:30:10 +08:00
Hongbo Wu	b036149fe2	fix feature tests	2023-10-05 14:30:10 +08:00
Hongbo Wu	d50174ef76	fix article tests	2023-10-05 14:30:10 +08:00
Hongbo Wu	669cddeec8	fix some rls issue	2023-10-05 14:30:10 +08:00
Hongbo Wu	cea5f790d8	fix some tests	2023-10-05 14:30:09 +08:00
Hongbo Wu	01c74cb7c4	fix tests for popular reads	2023-10-05 14:29:25 +08:00
Hongbo Wu	50bfc1fa74	fix tests	2023-10-05 14:29:23 +08:00
Hongbo Wu	bab957256d	fix multiple parameters in query builder	2023-10-05 14:28:52 +08:00
Hongbo Wu	e1a7a24323	improve search result	2023-10-05 14:28:52 +08:00
Hongbo Wu	1bade22076	fix file uploading	2023-10-05 14:28:52 +08:00
Hongbo Wu	e05d341952	fix file uploading	2023-10-05 14:28:52 +08:00
Hongbo Wu	86cff913fb	fix label search	2023-10-05 14:28:52 +08:00
Hongbo Wu	0747c4603c	fix bulk action	2023-10-05 14:28:52 +08:00
Hongbo Wu	afa1165a29	fix setting labels	2023-10-05 14:28:52 +08:00
Hongbo Wu	33ba17707b	fix setting labels	2023-10-05 14:28:52 +08:00
Hongbo Wu	0016b0a9a7	fix update library item	2023-10-05 14:28:52 +08:00
Hongbo Wu	d9d71aa443	fix highlights api	2023-10-05 14:28:52 +08:00
Hongbo Wu	5ed7e819e8	fix label constraint	2023-10-05 14:28:52 +08:00
Hongbo Wu	b72acf8e6f	fix permission issue	2023-10-05 14:28:52 +08:00
Hongbo Wu	4a55e801b4	fix typo	2023-10-05 14:28:50 +08:00
Hongbo Wu	3f572ea89a	fix typo	2023-10-05 14:27:52 +08:00
Hongbo Wu	65bb4aa85f	remove knex	2023-10-05 14:27:18 +08:00
Hongbo Wu	c1ad9b6f41	cont	2023-10-05 14:27:18 +08:00
Hongbo Wu	f8f3ec0c56	save claim in httpContext	2023-10-05 14:27:17 +08:00
Hongbo Wu	b1899e340d	replace library item	2023-10-05 14:25:10 +08:00
Hongbo Wu	c655f78f8c	replace highlight	2023-10-05 14:24:28 +08:00
Hongbo Wu	381e9bc173	replace database subscriber with db trigger	2023-10-05 14:24:05 +08:00
Hongbo Wu	612151c151	remove elastic from docker-compose	2023-10-05 14:24:04 +08:00
Hongbo Wu	5dce78eedb	update entity subscriber	2023-10-05 14:22:58 +08:00
Hongbo Wu	648a16615b	update search query	2023-10-05 14:22:58 +08:00
Hongbo Wu	751df68175	implement search function	2023-10-05 14:22:58 +08:00
Hongbo Wu	0b309b7a45	add recommendation db table and entity	2023-10-05 14:22:58 +08:00
Hongbo Wu	5f0e1af285	update highlight entity	2023-10-05 14:22:58 +08:00
Hongbo Wu	06000264d2	replace saveEmail	2023-10-05 14:22:58 +08:00
Hongbo Wu	fc52b522f2	fix test	2023-10-05 14:22:58 +08:00
Hongbo Wu	9370951893	create a repository directory	2023-10-05 14:22:56 +08:00
Hongbo Wu	425864ce01	remove knex	2023-10-05 14:21:56 +08:00
Hongbo Wu	928ed2e6a9	add libraryItemPreview entity class	2023-10-05 14:18:21 +08:00
Hongbo Wu	e1af53ab48	add libraryItem entity class	2023-10-05 14:18:21 +08:00
Hongbo Wu	789cb1d501	Merge pull request #2860 from omnivore-app/fix/self-hosting-login ... fix demo user not able to login when self hosting	2023-10-05 12:02:46 +08:00
Hongbo Wu	c008c3154d	make reminder task url optional env var	2023-10-05 11:36:34 +08:00
Hongbo Wu	16f72d6a74	fix labels and highlights were deleted when deleting items	2023-10-05 10:03:06 +08:00
Hongbo Wu	418f27f586	fix convert to email login does not change source field in user table	2023-10-04 16:35:50 +08:00
Jackson Harper	c41363b51c	Merge pull request #2831 from omnivore-app/feature/convert-to-email-api ... Update Email Feature	2023-10-04 14:30:35 +08:00
Hongbo Wu	3cf0c45fa9	mark unsubscribed newsletter in the db	2023-10-03 12:50:55 +08:00
Hongbo Wu	ee50529f95	add update email form on account page	2023-10-02 22:21:50 +08:00
Hongbo Wu	c40e64773a	convert to email api	2023-10-02 17:01:13 +08:00
Hongbo Wu	082e83b1ad	fix 406 not acceptable error from subscribing an rss feed	2023-10-02 10:52:50 +08:00
Hongbo Wu	010e2841da	fix getting 403 when parsing some rss url by setting a user agent in the request	2023-09-27 10:38:49 +08:00
Jackson Harper	21244a394d	Move debug directory to not interfere with usernames	2023-09-22 17:56:25 +08:00
Jackson Harper	53ac43563d	Set user properties instead of event properties on labels setter	2023-09-19 12:30:20 +08:00
Jackson Harper	6c6f27be3e	Properly tag user on labels event	2023-09-14 16:31:01 +08:00
Jackson Harper	e4b5bd6a67	Add code to join users without email on login	2023-09-14 11:24:19 +08:00
Jackson Harper	014614d097	Merge pull request #2759 from omnivore-app/feat/web-use-posthog ... Remove segment from the web	2023-09-13 14:01:08 +08:00
Jackson Harper	dfcfe64b1b	Add a create_user event	2023-09-13 13:44:26 +08:00
Jackson Harper	0946f3d3e8	Fix some console logs, better handle posthog	2023-09-13 12:52:34 +08:00
Jackson Harper	842e4bbf98	Update help links	2023-09-13 10:22:45 +08:00
Jackson Harper	293becf596	Ignore link density checks in newsletters	2023-09-12 15:53:43 +08:00
Jackson Harper	2a3214e393	Remove identify call from backend	2023-09-12 09:53:19 +08:00
Jackson Harper	a5eb527ae1	Merge pull request #2738 from omnivore-app/fix/api-enable-rules-notifications ... Remove debounce from notification rule	2023-09-07 18:07:06 +08:00
Jackson Harper	facde7d9b3	Remove import	2023-09-07 17:51:20 +08:00
Jackson Harper	e94529a96c	Merge pull request #2707 from Podginator/feat/saved-links ... Add Saved Searches, the ability to add new searches, delete searches, and reorder searches.	2023-09-07 17:27:48 +08:00
Jackson Harper	edd7246a9b	Remove debounce from notification rule	2023-09-07 17:24:49 +08:00
Jackson Harper	d0583bf271	Disable persisted queries in Apollo	2023-09-07 14:49:48 +08:00
Jackson Harper	f7e5a8cbfd	Increase unauthed rate limit	2023-09-07 14:01:18 +08:00
Thomas Rogers	79f42a5bde	Merge Main	2023-09-06 17:54:18 +02:00
Thomas Rogers	13ab13d94d	Code Review Comments	2023-09-06 17:51:42 +02:00
Jackson Harper	d2534d692b	Better preview URLs for onboarding articles	2023-09-04 18:27:56 +08:00
Thomas Rogers	faf7880fcd	Fix some bugs regarding scrolling, fix some UI Issues, and correct the name of Non-feed Filter.	2023-08-31 20:08:54 +02:00
Thomas Rogers	a58864fe2c	Merge main	2023-08-30 19:48:04 +02:00
Thomas Rogers	d433d48365	Add Filters, and editing filters.	2023-08-30 19:37:24 +02:00
Jackson Harper	79cea907a3	Fix typo with RSS typed subscription queries	2023-08-30 14:18:39 +08:00
Jackson Harper	fe3d3be7f5	Make query more clear	2023-08-29 20:17:13 +08:00
Jackson Harper	6c1484cd09	Add type filter	2023-08-29 20:06:28 +08:00
Jackson Harper	3565c7000e	Update subscriptions query	2023-08-29 17:03:49 +08:00
Jackson Harper	8851017100	Use andWhere query to not overwrite the where	2023-08-29 15:54:07 +08:00
Jackson Harper	89c17fd0ab	Only filter inactive newsletters out from the subscriptions response	2023-08-29 11:34:55 +08:00
Jackson Harper	49496f7aa2	Only return active subscriptions	2023-08-29 10:19:57 +08:00
Jackson Harper	feaa539c15	Use and where for type query. Prettier changes	2023-08-28 15:23:14 +08:00
Jackson Harper	aa0f4399aa	Remove prettier changes	2023-08-28 15:15:05 +08:00
Jackson Harper	38c9bbdcaa	Allow getting all subscritons regardless of type at the same time	2023-08-28 15:11:16 +08:00
Hongbo Wu	ec574d0ad9	use new color or the color of the last overlap highlight when merging highlights	2023-08-18 13:54:38 +08:00
Hongbo Wu	ce480806c0	add color to create/update/merge highlight api and highlight schema	2023-08-18 13:30:54 +08:00
Hongbo Wu	cd1fc5c4db	fix typo	2023-08-17 18:46:39 +08:00
Hongbo Wu	c45b760f71	fix test	2023-08-17 18:42:11 +08:00
Hongbo Wu	e54e61f3c0	reduce log size	2023-08-17 18:33:14 +08:00
Hongbo Wu	dedd76cfb7	do not publish a pubsub event if the page is imported	2023-08-17 18:24:57 +08:00
Hongbo Wu	ec88a9fe8e	we don't want to create thumbnail for imported pages	2023-08-17 18:05:51 +08:00
Hongbo Wu	0609043956	reduce retry	2023-08-17 17:59:24 +08:00
Hongbo Wu	6f315ef4d9	create a transaction for uploading files in the upload_files resolver	2023-08-16 21:19:03 +08:00
Hongbo Wu	755fb66588	Merge pull request #2651 from omnivore-app/feature/csv-importer ... feature/csv importer	2023-08-15 14:09:36 +08:00
Hongbo Wu	9ad9cea917	raise rss limit to 150	2023-08-14 22:27:43 +08:00
Hongbo Wu	5f6be169bd	add savedAt and publishedAt to saveUrl api	2023-08-14 17:10:34 +08:00
Hongbo Wu	fb845152d9	add header to the pocket data csv file	2023-08-14 13:44:30 +08:00
Hongbo Wu	93cca5040e	add comment	2023-08-11 14:41:21 +08:00
Hongbo Wu	05e73ebc9c	reduce readwise api timeout to 5 sec	2023-08-11 14:38:45 +08:00
Hongbo Wu	b18018ab2e	reduce max content size to 5MB and 10MB for both content and originalHtml	2023-08-11 14:02:29 +08:00
Hongbo Wu	792cf0b207	upgrade google cloud sdk	2023-08-11 13:55:29 +08:00
Hongbo Wu	55841e595f	fix dependecies	2023-08-11 11:12:03 +08:00
Hongbo Wu	61f553fd09	Merge pull request #2616 from omnivore-app/fix/thumbnail-delay ... fix thumbnail not pre-cached if it was already set	2023-08-10 18:45:48 +08:00
Hongbo Wu	7bd747601b	update thumbnail if failed to fetch and return url if already signed	2023-08-10 18:41:10 +08:00
Hongbo Wu	f06d2558c2	update thumbnail if failed to fetch and return url if already signed	2023-08-10 18:41:03 +08:00
Hongbo Wu	8809c9764a	max content length = 10MB	2023-08-10 18:12:29 +08:00
Hongbo Wu	fc76897df3	timeout = 5 when syncing omnivore data to readwise	2023-08-10 15:23:03 +08:00
Hongbo Wu	d4ac069503	fix webhook logs	2023-08-10 15:07:00 +08:00
Hongbo Wu	6acedb64c8	fix maximum call stack size error	2023-08-10 14:18:29 +08:00
Hongbo Wu	7506e27dd0	create or update an opt-in record with null grantedAt if exceeding max users	2023-08-10 13:47:39 +08:00
Hongbo Wu	163dbc73a6	do not send content in the thumbnail task payload	2023-08-10 11:55:59 +08:00
Hongbo Wu	51e0dfde23	set new limit = 1500	2023-08-10 09:43:40 +08:00
Hongbo Wu	3df0ffc5d3	fix existing opted in users not being able to be granted with the feature after increasing the limit	2023-08-08 22:08:37 +08:00
Hongbo Wu	3570079824	fix thumbnail not pre-cached if it was already set	2023-08-07 18:20:59 +08:00
Hongbo Wu	55e378c9b5	Merge pull request #2572 from omnivore-app/feature/pause-rss-feed ... add a button to pause rss feed	2023-08-03 13:33:11 +08:00
Hongbo Wu	f4e537a4c7	set max redirect = 5 and timeout = 30 seconds	2023-08-03 13:31:18 +08:00
Hongbo Wu	82d820ea8f	get link following the order of preference: via, self, alternate	2023-08-03 13:05:26 +08:00
Hongbo Wu	f3994a8a06	fix test	2023-08-02 22:00:09 +08:00
Hongbo Wu	b403dea247	add a button to pause rss feed	2023-08-01 22:07:53 +08:00
Jackson Harper	3d6d421e84	Merge pull request #2566 from omnivore-app/feat/web-improved-cards ... WIP: grid card design	2023-08-01 20:34:39 +08:00
Jackson Harper	c0dbbc3b86	Update icons	2023-08-01 12:31:12 +08:00
Hongbo Wu	00e7e49a4b	increase the max number of rss subscriptions to 50	2023-07-31 21:44:50 +08:00
Hongbo Wu	d0ad636b7f	use safe stringify in logs	2023-07-28 16:52:28 +08:00
Hongbo Wu	f20e12311a	check if the page is from rss feed	2023-07-28 13:53:19 +08:00
Hongbo Wu	9bc7ed92c0	we don't want to update an rss feed page if rss-feeder is tring to re-save it	2023-07-28 12:43:04 +08:00
Hongbo Wu	2e965afd69	reformat sql query in logs	2023-07-27 17:34:12 +08:00
Hongbo Wu	99a52f8d56	replace all the console logs with logger logs	2023-07-27 16:06:44 +08:00
Hongbo Wu	808f6f3d3a	remove trace	2023-07-26 22:00:37 +08:00
Jackson Harper	5bf3213e49	Dont send newsletter notifications	2023-07-26 20:53:50 +08:00
Hongbo Wu	61967fdde5	fix test	2023-07-26 18:07:25 +08:00
Hongbo Wu	ced1a711eb	fix test	2023-07-26 17:12:40 +08:00
Hongbo Wu	7b36156ce6	add a trace when log size too big	2023-07-26 17:07:24 +08:00
Hongbo Wu	8cd47eec11	run prettier	2023-07-26 16:41:55 +08:00
Hongbo Wu	07d43812a7	use winston logger instead of console log so the log will be truncated if it exceeds the max entry size	2023-07-26 16:27:18 +08:00
Hongbo Wu	66490df72f	deeply truncate log object size	2023-07-26 15:52:13 +08:00
Hongbo Wu	f0e62b1d2a	use lodash instead of standalone lodash.debounce package	2023-07-26 13:41:51 +08:00
Hongbo Wu	0a227decdd	Merge pull request #2515 from omnivore-app/fix/push-notification ... fix/push notification	2023-07-26 12:36:13 +08:00
Hongbo Wu	2841783504	limit max log entry size to 256kb	2023-07-25 23:39:03 +08:00
Hongbo Wu	f5ae16a904	disable gql request log	2023-07-25 23:03:57 +08:00
Hongbo Wu	ecd6a2922f	disable sentry error handler	2023-07-25 22:32:00 +08:00
Hongbo Wu	fd6223ee93	revert changes	2023-07-25 21:20:26 +08:00
Hongbo Wu	a3e86a9bc1	only log error.message	2023-07-25 20:52:15 +08:00
Hongbo Wu	0e38060b0c	set maxEntrySize for logger to 256KB	2023-07-25 20:07:02 +08:00
Hongbo Wu	1006cdad36	reduce error log size	2023-07-25 19:11:08 +08:00
Hongbo Wu	de647e44bd	Revert "move sentry error handler after apollo middleware" ... This reverts commit 8d8d2b592c.	2023-07-25 19:10:12 +08:00
Hongbo Wu	8d8d2b592c	move sentry error handler after apollo middleware	2023-07-25 18:18:40 +08:00
Hongbo Wu	77456373bf	return content without highlight if it takes too long to process highlighted content ... * timeout if the get text node function takes more than 10 minutes	2023-07-25 11:19:06 +08:00
Jackson Harper	fa1ff9ba17	Upgrade node to 18.16	2023-07-24 15:55:11 +08:00
Hongbo Wu	0d3b742837	limit 20k articles per pocket import	2023-07-21 18:03:57 +08:00
Hongbo Wu	e0a6a2e0a4	limit number of rss subscriptions to 20	2023-07-21 15:28:20 +08:00
Hongbo Wu	99b12c6cfb	debounce send push notificaiton within 1 minute	2023-07-20 21:48:30 +08:00
Hongbo Wu	9a224d2324	add event_types and fix the validation of form on rules page	2023-07-20 18:33:30 +08:00
Hongbo Wu	4dad4a6fa3	revert push notification	2023-07-20 17:29:26 +08:00
Hongbo Wu	6004b51956	return not found if rss feed not found	2023-07-20 14:21:36 +08:00
Hongbo Wu	7ceb96a93e	fix edit rss ui	2023-07-20 13:35:43 +08:00
Hongbo Wu	400739f61c	stop writing empty csv file if no new data retrieved from the integration	2023-07-20 10:32:46 +08:00
Hongbo Wu	6ef74a2601	disable import button while importer is running	2023-07-20 09:52:23 +08:00
Hongbo Wu	cd440a77c3	send last fetched at as a unix timestamp in ms	2023-07-19 14:09:38 +08:00
Hongbo Wu	35c074e647	create a cloud task to fetch rss feed item for the new subscription	2023-07-19 10:07:08 +08:00
Hongbo Wu	dcb306bcfc	do not include archived items in library or subscription	2023-07-18 16:45:24 +08:00
Hongbo Wu	456a73f4ab	make library an internal label	2023-07-17 18:29:35 +08:00
Hongbo Wu	44c9bfce48	Merge pull request #2494 from omnivore-app/fix/rss-feed ... fix/rss feed	2023-07-17 17:50:00 +08:00
Hongbo Wu	3cfd390346	Merge pull request #2489 from omnivore-app/more-search-scope ... add in:subscription and in:library in search scope	2023-07-17 17:28:24 +08:00
Hongbo Wu	03ce4b00d2	add authorization header in rss feed fetching request	2023-07-17 17:27:12 +08:00
Hongbo Wu	be1e8169ba	exclude items with newsletter or rss labels in library	2023-07-17 17:06:39 +08:00
Hongbo Wu	113720caf6	wrap document fragment in the <html> and <body> in order to parse the HTML correctly	2023-07-17 16:14:59 +08:00
Hongbo Wu	b3dcf0ee46	default return builder	2023-07-17 16:14:03 +08:00
Hongbo Wu	e3af3f526c	add in:subscription and in:library in search scope	2023-07-14 17:07:25 +08:00
Hongbo Wu	153662ce6a	set savedAt and publishedAt timestamp for rss item	2023-07-14 12:21:34 +08:00
Hongbo Wu	983f0dcb99	add rss to the search scope	2023-07-13 11:09:13 +08:00
Hongbo Wu	9697ae2eca	make rss an internal label	2023-07-13 10:58:37 +08:00
Hongbo Wu	87275fbb9c	use a separate queue for fetching rss feed item content	2023-07-13 10:27:15 +08:00
Hongbo Wu	3444480b95	change the rss-feed cronjob api endpoint	2023-07-12 10:53:32 +08:00
Hongbo Wu	435fa0c541	Merge pull request #2478 from omnivore-app/fix/puppeteer-timezone ... add optional locale and timezone to saveUrl request for puppeteer to use	2023-07-11 17:32:10 +08:00
Hongbo Wu	1f283e6122	add optional locale and timezone to saveUrl request for puppeteer to use	2023-07-11 16:15:32 +08:00
Hongbo Wu	b1b4a2e3fd	save rss feed url in the index	2023-07-11 13:14:43 +08:00
Hongbo Wu	c500997693	add lastFetchedAt to the cloud task payload	2023-07-11 13:05:27 +08:00
Hongbo Wu	eb9a3eddd0	skip old item	2023-07-11 12:59:55 +08:00
Hongbo Wu	44473ba089	upload feed subscriptions in cloud storage in cronjob	2023-07-11 12:22:49 +08:00
Hongbo Wu	56a27878dc	add updateSubscription api	2023-07-10 15:24:33 +08:00
Hongbo Wu	68667053c8	create an endpoint to fetch all rss feeds	2023-07-07 21:15:42 +08:00
Hongbo Wu	574636451d	update api to accommodate rss feed subscriptions	2023-07-06 19:51:30 +08:00
Hongbo Wu	7075375e2b	fix test	2023-07-06 19:51:30 +08:00
Hongbo Wu	2adf753f8f	update graphql schema	2023-07-06 19:51:30 +08:00
Hongbo Wu	a6192ed86b	add rss subscription entity class	2023-07-06 19:51:30 +08:00
Jackson Harper	0c96172053	Merge pull request #2458 from omnivore-app/feature/search-filter-cat ... feat: add category field to Filter object	2023-07-05 10:41:43 -07:00
Hongbo Wu	599d61016e	feat: add category field to Filter object	2023-07-03 19:06:45 +08:00
Hongbo Wu	e4284d215c	feat: boost sitename score by 2 times in the basic search	2023-07-03 18:09:41 +08:00
Hongbo Wu	afaaf8af60	feat: allow searching by note in the basic search	2023-07-03 18:06:54 +08:00
Hongbo Wu	a413521e3b	Merge pull request #2445 from omnivore-app/feature/create-labels ... feat: create labels if not exist when setting labels	2023-07-03 16:54:15 +08:00
Hongbo Wu	88dc514d27	feat: allow recovering a soft deleted item through updatePage api	2023-06-30 18:39:43 +08:00
Hongbo Wu	ba35fa4447	feat: show soft deleted pages within 14 days by searching in:trash	2023-06-30 18:05:41 +08:00
Hongbo Wu	6be9afb619	feat: allow searching by note:	2023-06-30 16:02:49 +08:00
Hongbo Wu	53b854b69f	fix: improve full-text search result ... * always sort by score first * boost title score by 3 times * tie breaker is 0.3	2023-06-30 15:32:09 +08:00
Hongbo Wu	13afb7febd	feat: create labels if not exist when setting labels in the highlight	2023-06-30 11:40:39 +08:00
Hongbo Wu	8d3cac989b	feat: create labels if not exist when setting labels in the page	2023-06-30 11:08:03 +08:00
Hongbo Wu	74f2dd6bf1	catch authorization error in rate limiter and 5 rpm	2023-06-28 17:04:47 +08:00
Hongbo Wu	b7c133d58e	fix: email sending and recommendation ... * use token in the custom header as the key in rate limiter	2023-06-28 17:00:11 +08:00
Hongbo Wu	b60ace4877	log unauthorized error	2023-06-28 12:22:32 +08:00
Hongbo Wu	1b069c2696	catch authorization error and return 401	2023-06-28 12:19:47 +08:00
Hongbo Wu	af7350bd31	fix: importer status update email not sent ... * create omnivore-authorization header to contain the token	2023-06-28 11:00:40 +08:00
Hongbo Wu	7c0c4eaa40	fix: type error	2023-06-27 21:05:00 +08:00
Hongbo Wu	ed8287df19	remove tracking params from tweet url	2023-06-27 17:49:31 +08:00
Hongbo Wu	bb359d9747	save normalized url	2023-06-27 16:58:57 +08:00
Hongbo Wu	f372636d6e	fix: tweet not saved correctly when using share button on iOS	2023-06-27 16:47:36 +08:00
Hongbo Wu	90444788ba	Merge pull request #2400 from omnivore-app/feat/web-multi-select-labels ... Add support for multi select Add Label	2023-06-22 09:42:59 +08:00
Jackson Harper	cceff32e5f	Return non-error if setting labels is a noop	2023-06-21 20:24:36 +08:00
Hongbo Wu	826565a2c4	temporarily skip unsubscribing by url and mark them as unsubscribed automatically	2023-06-21 12:13:22 +08:00
Hongbo Wu	4f74b32ff6	do not subscribe a newsletter if subscription already exists and is unsubscribed	2023-06-21 12:02:15 +08:00
Hongbo Wu	84788e7551	fix: mark failed to unsubscribe newsletters as unsubscribed state	2023-06-21 11:33:49 +08:00
Hongbo Wu	ea6d06f464	Merge pull request #2375 from omnivore-app/fix/pocket-importer ... fix pocket paginated api	2023-06-19 11:19:31 +08:00
Hongbo Wu	0d8a3f7346	fix pocket paginated api	2023-06-16 18:11:31 +08:00
Hongbo Wu	c8e3f79655	Merge pull request #2368 from omnivore-app/fix/api-robots-txt ... Add robots.txt to the list of reserved names	2023-06-16 16:38:06 +08:00
Jackson Harper	cf7c50468e	Use normalized URLs when querying article saving requests ... When we save pages we save the normalized URL, so if a user saves a URL like https://foo.com/ in elastic it will be saved as https://foo.com but the browser will redirect to this url: https://omnivore.app/article?url=https://foo.com/ and the URL lookup will fail.	2023-06-16 15:15:24 +08:00
Jackson Harper	5dd28de9e8	Add security.txt file too	2023-06-16 12:54:30 +08:00
Jackson Harper	52af2d8cb4	Add robots.txt to the list of reserved names	2023-06-16 12:54:30 +08:00
Hongbo Wu	9a3673e315	Merge pull request #2367 from omnivore-app/fix/refresh-index ... refresh index after altering the page state or adding/removing/updating labels/highlights	2023-06-15 12:18:53 +08:00
Hongbo Wu	ceaa7b0356	refresh index after altering the page state or adding/removing/updating labels/highlights	2023-06-15 11:37:09 +08:00
Hongbo Wu	5d2ab3f554	create another rate limiter for auth api with 4 rpm	2023-06-14 19:31:18 +08:00
Hongbo Wu	fc08a434e1	Merge pull request #2359 from omnivore-app/fix/pocket-import ... fix/pocket import	2023-06-14 15:32:10 +08:00
Jackson Harper	aaf6bb617f	Remove backend events that we dont need to watch	2023-06-14 13:19:39 +08:00
Hongbo Wu	11cf9f7c28	stringify csv before uploading	2023-06-14 12:50:06 +08:00
Hongbo Wu	d851a705a6	refresh index after archiving page	2023-06-13 12:44:32 +08:00
Hongbo Wu	771114b30d	refresh index if update is not async	2023-06-13 12:43:18 +08:00
Hongbo Wu	1a48ad03e4	use a separate queue for thumbnailer	2023-06-12 12:25:04 +08:00
Hongbo Wu	8904247620	add test	2023-06-09 18:58:41 +08:00
Hongbo Wu	b12e127d15	use gold color for the label	2023-06-09 17:36:38 +08:00
Hongbo Wu	5db4354f3f	make internal optional in label response	2023-06-09 17:33:14 +08:00
Hongbo Wu	bd4204fc42	add analytics	2023-06-09 13:14:14 +08:00
Hongbo Wu	55904ccf04	feat: add set favorite article api which adds the favorites internal label to it	2023-06-09 13:09:47 +08:00
Hongbo Wu	6116746e80	feat: make newsletters and favorites internal labels	2023-06-09 12:48:46 +08:00
Hongbo Wu	51032d2bf1	fix: test	2023-06-08 18:52:47 +08:00
Hongbo Wu	039f00e3b2	perf: add max doc and throttling to bulk api	2023-06-08 18:36:09 +08:00
Hongbo Wu	4678d6f473	check label uniqueness by name	2023-06-08 13:33:59 +08:00
Hongbo Wu	3630d743d7	add unique labels	2023-06-08 13:29:05 +08:00
Hongbo Wu	de3623d3f2	allow mark as read and add labels bulk actions	2023-06-08 13:05:30 +08:00
Hongbo Wu	1af845e4e5	feat: add bulk action api schema	2023-06-07 21:50:43 +08:00
Hongbo Wu	b698029972	store token in cookie instead of authorization header	2023-06-07 14:24:52 +08:00
Hongbo Wu	c0872939a7	Merge pull request #2311 from omnivore-app/feature/metadata ... Feature: Set thumbnail and pre-cache all the images in the content	2023-06-07 12:08:20 +08:00
Hongbo Wu	9be84fb473	create thumbnail task for newsletters too	2023-06-07 11:52:13 +08:00
Hongbo Wu	986c4eb48b	fix: parse unsubscribe mailto email address and set subject	2023-06-06 19:53:18 +08:00
Hongbo Wu	4f4f50af49	fix: redundant argument	2023-06-06 12:05:33 +08:00
Hongbo Wu	0c3fa2cdd4	feat: start a cloud task to set thumbnail and pre-cache all the images in the content	2023-06-06 11:56:31 +08:00
Hongbo Wu	e2315b5662	feat: add thumbnail handler	2023-06-06 11:02:46 +08:00
Hongbo Wu	c66d13177f	Merge pull request #2295 from omnivore-app/fix/twitter-handler-timeout ... Feature: importer metrics collector and fix puppeteer timeout issue	2023-06-05 17:17:50 +08:00
Jackson Harper	e53cc28683	Fix issue where we return EPUB content reader for items marked as book in metadata ... If we parse a page with <meta property="og:type" content="book"> in it now, we incorrectly set this to ContentReader.EPUB, which then causes syncing issues on iOS which doesn't have an EPUB reader type defined in its GraphQL schema yet.	2023-06-05 14:44:51 +08:00
Hongbo Wu	a439aac7e7	feat: update api to include preview image	2023-06-05 12:38:30 +08:00
Hongbo Wu	d69e732579	fix: dedupe url if failed to save	2023-06-01 16:11:35 +08:00
Hongbo Wu	15b41560de	fix: improve the speed of generating highlighted markdown content	2023-05-26 15:51:54 +08:00
Hongbo Wu	15b977f7c6	fix: set server timeout = 10 mins	2023-05-26 15:00:46 +08:00
Jackson Harper	751aec2676	Use number instead of function	2023-05-19 18:10:28 +08:00
Jackson Harper	f671b3625c	Change rate limiting rules	2023-05-19 18:05:57 +08:00
Hongbo Wu	4cc9547935	fix: fallback to html if failed to convert html to markdown	2023-05-19 16:55:50 +08:00
Hongbo Wu	87f55eabf6	fix: sentry report on node is null	2023-05-19 10:54:43 +08:00
Hongbo Wu	331ddf69d4	remove debugging log	2023-05-18 21:12:44 +08:00
Hongbo Wu	0adab7ad23	fix: sentry report on unable to find highlight	2023-05-18 21:07:51 +08:00
Hongbo Wu	48554be76d	fix: wrap link with ==	2023-05-18 21:04:27 +08:00
Hongbo Wu	2bf932cbdc	fix: &npsp; breaks highlight	2023-05-18 16:50:49 +08:00
Jackson Harper	058ccf3006	Merge pull request #2183 from omnivore-app/fix/unauthed-rate-limit ... Reduce the rate limit for unauthenticated users	2023-05-18 15:15:18 +08:00
Jackson Harper	1d37f04808	Properly call getClaimsByToken	2023-05-16 17:14:38 +08:00
Hongbo Wu	e14080a05f	Set labels = undefined if pocket tags is undefined	2023-05-16 11:04:22 +08:00
Hongbo Wu	62ce7ff713	Merge pull request #2200 from omnivore-app/fix/pocket-import ... Pass pocket auth token in query string	2023-05-15 21:53:15 +08:00
Hongbo Wu	175b82485f	Fix typo	2023-05-15 17:39:29 +08:00
Hongbo Wu	c2e62155cc	Pass pocket auth token in query string	2023-05-15 16:56:27 +08:00
Hongbo Wu	e3d8fae6c5	Remove debug log	2023-05-12 19:28:19 +08:00
Hongbo Wu	791d11f581	Allow content to be converted into highlightedMarkdown format	2023-05-12 19:27:04 +08:00
Jackson Harper	c49ca0149f	Set max to 10/minute for unauthenticated requests	2023-05-12 17:11:20 +08:00
Hongbo Wu	565c932a2d	Add archivedAt to the search item in the search api response	2023-05-12 12:22:11 +08:00
Jackson Harper	cbdd4f0a71	Merge pull request #2184 from omnivore-app/fix/rm-auth-cookie-on-reset ... Clear auth cookie when password reset is triggered	2023-05-11 17:33:16 +08:00
Jackson Harper	8d9f54c62c	Fix linting	2023-05-11 13:36:27 +08:00
Jackson Harper	78681df4ab	Use clearCookie, rm the pending auth cookie too	2023-05-11 13:32:43 +08:00
Jackson Harper	4329c28904	On password reset clear the auth cookie	2023-05-11 13:29:35 +08:00
Jackson Harper	88aa7aad4c	Reduce the rate limit for unauthenticated users	2023-05-11 13:23:43 +08:00
Jackson Harper	f8df71508b	Enforce max lengths on signup credentials	2023-05-10 23:09:31 +08:00
Hongbo Wu	a4d385bc83	add sorting by wordsCount	2023-05-03 08:36:33 +08:00
Hongbo Wu	31cbcc7c7f	validate sign up request body in mobile sign-up router	2023-04-27 12:18:32 +08:00
Hongbo Wu	0e8887a1d9	trim user email before saving in db	2023-04-27 11:39:34 +08:00
Hongbo Wu	673c43aee0	trim user email in mobile sign-in router	2023-04-27 11:37:27 +08:00
Hongbo Wu	fb4c8cb1ed	trim user email in mobile sign-up router	2023-04-27 11:33:03 +08:00
Hongbo Wu	692fd7b959	Fix save url error when uid is not in the apollo context	2023-04-25 17:01:25 +08:00
Hongbo Wu	3841734deb	Fix forwarded email subject being treated as an URL, for example "Fwd: subject"	2023-04-24 21:16:44 +08:00
Hongbo Wu	b891e547c9	Make content optional	2023-04-24 21:03:39 +08:00
Hongbo Wu	e512ba0bad	Fix tests	2023-04-24 18:04:18 +08:00
Hongbo Wu	69968e5288	Save url if the email subject is a parsable url	2023-04-24 12:54:23 +08:00
Hongbo Wu	bce7afde50	Eagerly load profile of the user	2023-04-24 12:53:40 +08:00
Hongbo Wu	b57ccd0616	Allow wildcards in query filter	2023-04-21 17:39:04 +08:00
Hongbo Wu	b2c115f9d9	Allow wildcards in label: filter	2023-04-21 17:10:46 +08:00
Hongbo Wu	130677b50c	Add contentReader to the typeAheadSearchResult	2023-04-20 10:00:56 +08:00
Hongbo Wu	5ce4bf7171	Compare labels with lower cased	2023-04-20 09:38:04 +08:00
Hongbo Wu	62b772eeca	Allow saving epub from email attachments	2023-04-19 11:30:49 +08:00
Jackson Harper	3c12162866	Fix typo in content type check	2023-04-18 11:22:44 +08:00
Jackson Harper	69df32d428	Implement a reader for epubs	2023-04-18 09:00:48 +08:00
Hongbo Wu	e6690709c3	Replace express-http-context with express-http-context2	2023-04-12 17:50:13 +08:00
Hongbo Wu	014dc773e3	Fix labels not saved correctly by saveUrl	2023-04-12 17:04:59 +08:00
Hongbo Wu	874247661b	resolve conflicts	2023-04-12 17:04:59 +08:00
Hongbo Wu	c07ada0218	resolve conflicts	2023-04-12 17:04:59 +08:00
Hongbo Wu	42dc3f7330	Fix some external requests not being mocked	2023-04-12 17:04:59 +08:00
Hongbo Wu	9857af27e2	Add labels and state to createArticle API	2023-04-12 17:04:59 +08:00
Hongbo Wu	abd42f7064	Add labels and state to saveUrl API	2023-04-12 16:50:11 +08:00
Hongbo Wu	ce273b5172	Add test case for archives page and saves labels in savePage API	2023-04-12 16:50:11 +08:00
Hongbo Wu	87d3f5b3d8	Add labels and state to savePage API	2023-04-12 16:50:11 +08:00
Hongbo Wu	fcf08fc364	Add state and labels to savePageInput	2023-04-12 16:50:11 +08:00
Hongbo Wu	4b578bbebb	Process labels and state in the csv file	2023-04-12 16:50:11 +08:00
Hongbo Wu	9a46e935d9	Add [] to the labels in csv	2023-04-12 16:50:11 +08:00
Hongbo Wu	06b6583f29	Mock cloud storage bucket	2023-04-12 16:50:11 +08:00
Hongbo Wu	76b63efdc1	Fix tests	2023-04-12 16:50:11 +08:00
Hongbo Wu	10c01c12f2	Add test case for integration import task handler	2023-04-12 16:50:10 +08:00
Hongbo Wu	56fd3d7617	Create a router to handle importing from integrations cloud task	2023-04-12 16:50:10 +08:00
Hongbo Wu	503ec3d413	Update task name	2023-04-12 16:50:10 +08:00
Hongbo Wu	9ff0711a73	Remove wrong comments	2023-04-12 16:50:10 +08:00
Hongbo Wu	c8ec84562b	Create an import from integration cloud task	2023-04-12 16:50:10 +08:00
Hongbo Wu	b62b769604	Paginate pocket api	2023-04-12 16:50:10 +08:00
Hongbo Wu	1c5991aded	import from integration api return a boolean value	2023-04-12 16:50:10 +08:00
Hongbo Wu	c78fe9f87d	Paginate pocket retrieve api response data	2023-04-12 16:50:10 +08:00
Hongbo Wu	737957ff82	Add import from pocket to web	2023-04-12 16:50:10 +08:00
Hongbo Wu	2e5e3bd19e	Add headers	2023-04-12 16:50:10 +08:00
Hongbo Wu	59621768c8	Get request token from pocket	2023-04-12 16:50:10 +08:00
Hongbo Wu	ae2f695df8	Get access token from pocket	2023-04-12 16:50:10 +08:00
Hongbo Wu	9c7f6fefca	Add PocketIntegration service to the list	2023-04-12 16:50:10 +08:00
Hongbo Wu	c3bfb593c6	Fix regression	2023-04-12 16:50:10 +08:00
Hongbo Wu	02247bc0a6	Add import from integration api implementation	2023-04-12 16:50:10 +08:00
Hongbo Wu	e2d050288e	write the list of urls to a csv file and upload it to gcs	2023-04-12 16:50:10 +08:00
Hongbo Wu	94d8903ec1	Add pocket integration support	2023-04-12 16:50:10 +08:00
Hongbo Wu	0bf4119c92	Refactor with factor pattern	2023-04-12 16:50:10 +08:00
Hongbo Wu	4ce4cd0a62	Add a type for export or import to the integration table	2023-04-12 16:50:10 +08:00
Hongbo Wu	9056318667	Change type field in integration table from enum to text type	2023-04-12 16:50:10 +08:00
Hongbo Wu	b43f6904f4	Merge pull request #2021 from omnivore-app/fix/subscriptions ... creates or updates subscription only if their is a valid unsubscribe link	2023-04-12 13:17:34 +08:00
Hongbo Wu	80d0e7c944	Move pageType to function_resolver	2023-04-12 12:37:22 +08:00
Hongbo Wu	158a75807c	Use camel case	2023-04-12 12:27:05 +08:00
Hongbo Wu	4969391d4c	Replace picture with image	2023-04-12 12:15:53 +08:00
Hongbo Wu	c5eb223c07	Add tweet, video, picture pageType	2023-04-12 11:53:23 +08:00
Hongbo Wu	f28912728a	creates or updates subscription only if their is a valid unsubscribe link	2023-04-11 22:39:47 +08:00
Hongbo Wu	83cd0ac880	Return error if query exceeds 255 char	2023-04-06 12:03:26 +08:00
Hongbo Wu	3d599ad7ab	Remove sanitize directive on query argument because we will change the argument type if doing so	2023-04-06 11:56:19 +08:00
Hongbo Wu	c596e88f46	Set search query max size to 255 char	2023-04-05 12:20:52 +08:00
Hongbo Wu	e5ffd3f5bd	Stop inserting search history to the DB in the search API	2023-04-05 11:45:39 +08:00
Hongbo Wu	819d275497	Add wordsCount if undefined when creating pages in elastic	2023-04-04 12:52:46 +08:00
Hongbo Wu	32d31af847	Merge pull request #1970 from omnivore-app/double-quote-size-limit ... Double max length of quote in highlights	2023-04-04 11:44:28 +08:00
Hongbo Wu	c52d2f0f56	Fix pageType to be null in updatesSince api response	2023-04-04 11:00:13 +08:00
Hongbo Wu	8d79f858c3	Double max length of quote in highlights	2023-04-03 18:46:59 +08:00
Hongbo Wu	53a2df5dd2	Fix start_date or end_date is undefined in elasticsearch search query	2023-03-27 18:45:24 +08:00
Hongbo Wu	dd24cfbeec	Remove async	2023-03-23 10:45:39 +08:00
Hongbo Wu	abb8df26e5	Set HIGHLIGHT as default highlight type in every highlight	2023-03-23 10:44:30 +08:00
Hongbo Wu	aae16d7a4d	Set HIGHLIGHT as default highlight type	2023-03-22 22:02:03 +08:00
Hongbo Wu	b3a965781c	resolve conflicts	2023-03-22 10:40:03 +08:00