1774 commits

Author	SHA1	Message	Date
Jackson Harper	dde80091b4	Merge pull request #3494 from omnivore-app/fix/rss-more-content-fetch-blocks ... Add rss.arxiv and daringfireball to content fetch block lists	2024-02-03 10:57:04 +08:00
Jackson Harper	7fffa8ce75	Add some logging	2024-02-03 10:04:40 +08:00
Jackson Harper	46affc0508	Add xkcd	2024-02-03 10:02:25 +08:00
Jackson Harper	36915ec1ec	Add rss.arxiv and daringfireball to content fetch block lists	2024-02-03 10:01:18 +08:00
Jackson Harper	7937c73de1	Remove old warning thats no longer needed	2024-02-03 09:54:54 +08:00
Jackson Harper	9102fc9ae4	Merge pull request #3491 from omnivore-app/fix/dont-sync-readingtime-too-soon ... Only sync reading times if they are more than a minute old	2024-02-03 09:09:52 +08:00
Jackson Harper	a745e72ce4	Allow setting postgres extra options via envvar	2024-02-03 08:57:41 +08:00
Jackson Harper	9cc046e3a2	Only sync reading times if they are more than a minute old ... this should hopefully reduce contention with other rules running	2024-02-02 20:41:14 +08:00
Hongbo Wu	b68adc5983	fix tests	2024-02-02 17:05:09 +08:00
Hongbo Wu	df2dbb2916	fix: find library item by url not using correct index	2024-02-02 16:46:13 +08:00
Hongbo Wu	74783313da	fix query	2024-02-02 16:18:41 +08:00
Hongbo Wu	1742ffda24	change priority to 5	2024-02-02 16:18:41 +08:00
Hongbo Wu	25e374f6ff	fix: replace webhook endpoint with a bullmq job	2024-02-02 16:18:40 +08:00
Hongbo Wu	5e49131fc2	Merge pull request #3470 from omnivore-app/fix/async-bulk-action ... async bulk action on item count > 100 and create batch jobs	2024-02-02 16:00:12 +08:00
Hongbo Wu	54a4022409	reduce size	2024-02-02 15:45:11 +08:00
Hongbo Wu	e980be7771	skip test	2024-02-02 14:14:41 +08:00
Hongbo Wu	26b7103f0b	improve batch update	2024-02-02 13:13:18 +08:00
Jackson Harper	d339c5e8f1	Use the md5 index when looking up library items by original url	2024-02-02 13:08:00 +08:00
Hongbo Wu	2d1eebfad8	remove size	2024-02-02 12:21:08 +08:00
Hongbo Wu	9c28726a9b	add parent config	2024-02-02 11:49:02 +08:00
Hongbo Wu	1bccf33320	add job_id and parent for tracking	2024-02-02 10:50:07 +08:00
Jackson Harper	ed7ac6c1b4	Fix typo in lifecycle endpoint path	2024-02-01 20:25:52 +08:00
Hongbo Wu	2a0e6f8fa4	fix tests	2024-02-01 17:44:32 +08:00
Hongbo Wu	702bd6c3c6	async bulk action on item count > 100 and create batch jobs	2024-02-01 17:44:30 +08:00
Jackson Harper	7f4f7c6a80	Merge pull request #3479 from omnivore-app/fix/queue-processor-lifecycle-hook ... Implement a lifecycle hook for queue processor so it can shut down gracefully	2024-02-01 17:01:11 +08:00
Hongbo Wu	8ac696e87d	Merge pull request #3480 from omnivore-app/fix/mark-read-action ... fix: update reading progress in cache when mark as read action was triggered by a rule	2024-02-01 16:56:21 +08:00
Hongbo Wu	252443f1e4	fix: update reading progress in cache when mark as read action was triggered in a rule	2024-02-01 16:45:08 +08:00
Jackson Harper	0e9e331e18	Implement a lifecycle hook for queue processor so it can shut down gracefully	2024-02-01 16:43:57 +08:00
Jackson Harper	3feef7ca66	Use the DB stored reading progress when calculating maxes	2024-02-01 16:23:21 +08:00
Jackson Harper	f03b7182b3	Merge pull request #3460 from omnivore-app/feat/reading-progress-write-behind ... Write behind cache for reading progress	2024-02-01 14:43:45 +08:00
Hongbo Wu	abaf726044	revert the change of priority	2024-02-01 13:22:06 +08:00
Hongbo Wu	19fe60d27a	change the update-db job priority to 5 and use exponential backoff strategy for retrying	2024-02-01 13:14:01 +08:00
Hongbo Wu	e13c418389	remove the inner join with highlights table in the sql of update-label job	2024-02-01 13:09:50 +08:00
Jackson Harper	02c28be403	reduce unction is no longer async	2024-02-01 11:24:13 +08:00
Jackson Harper	792f13edfc	Remove async from non-async function	2024-02-01 10:49:48 +08:00
Jackson Harper	1b44cb860b	Function for cron jobs dont include queue in the read position metrics since its not part of that queue	2024-02-01 10:33:51 +08:00
Jackson Harper	7d6645b87a	Rebase	2024-02-01 10:02:38 +08:00
Jackson Harper	570b7a4b78	Use a set in redis and push items to the set ... This lets us grab all available items, apply them, and then remove them from the set. Once completed the items that were applied can be removed from the set, if any new items were added during that time they wont be effected.	2024-02-01 09:59:42 +08:00
Jackson Harper	0e37bfa03d	We need to update readAt because that isnt resolved from the cache	2024-02-01 09:59:42 +08:00
Jackson Harper	c47a720fe4	Do not cache on write, so the most updated item is always available on read ... This is mostly an issue on tests where all the queries can be run in a single process, but if we cache on write, it means the next read will just have the cached value, instead of the calculated maximum value.	2024-02-01 09:59:42 +08:00
Jackson Harper	bdf01ddaab	Clear cache on force, return updated read position values	2024-02-01 09:59:42 +08:00
Jackson Harper	a060aaf496	Make reading position parsing safer	2024-02-01 09:59:42 +08:00
Jackson Harper	9dc070243a	Revert temp	2024-02-01 09:59:41 +08:00
Jackson Harper	d8ea4ff1c5	Add reading position messages to exported metrics for queue processor	2024-02-01 09:59:41 +08:00
Jackson Harper	7bb3718a8f	Add job to sync read position data	2024-02-01 09:59:41 +08:00
Jackson Harper	a40f5bed55	Create a service for interacting with cached read positions	2024-02-01 09:59:41 +08:00
Jackson Harper	fbfa934479	Write behind reading progress as a list that can be reduced at commit or read time	2024-02-01 09:59:39 +08:00
Jackson Harper	c2327781e1	Remove empty ctor	2024-02-01 09:57:11 +08:00
Jackson Harper	86c80d991f	Write behing cache for reading progress	2024-02-01 09:57:11 +08:00
Hongbo Wu	c3276fdaf1	Merge pull request #3466 from omnivore-app/fix/rewrite-trigger-functions-in-db ... create a job for db trigger	2024-01-31 19:47:45 +08:00