9279 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Jackson Harper	2f405ebed0	Use savePage service instead of API call	2024-01-20 14:07:22 +08:00
Jackson Harper	3a18327230	Remove unused	2024-01-20 13:43:17 +08:00
Jackson Harper	7786221fbc	Fix update_subscription awaits	2024-01-20 13:38:25 +08:00
Jackson Harper	89537c13de	Simplify function input	2024-01-20 13:32:11 +08:00
Jackson Harper	1898e32607	Use service instead of API to update subscription	2024-01-20 13:27:54 +08:00
Jackson Harper	e5dec228f3	Remove some debug	2024-01-20 12:46:22 +08:00
Jackson Harper	4eeb012b78	Pass lits of fetchContentTasks into rss handlers ... This prevents the list from growing on each run	2024-01-20 11:36:14 +08:00
Jackson Harper	2b536c6086	Add more debug	2024-01-20 10:48:57 +08:00
Jackson Harper	52050728ba	Add more debugging logs	2024-01-20 10:29:07 +08:00
Jackson Harper	f4af4593c2	Fix debug line	2024-01-20 10:04:54 +08:00
Jackson Harper	4c1a182c7f	Add a refresh context to make debugging refresh runs easier	2024-01-20 09:50:21 +08:00
Jackson Harper	2961f69f02	Add a refresh context to make debugging rss jobs easier	2024-01-20 09:47:46 +08:00
Jackson Harper	a9eeda9369	Remove non-useful logging line	2024-01-20 09:23:45 +08:00
Jackson Harper	e9f22bba43	Some debug	2024-01-20 09:09:25 +08:00
Hongbo Wu	fb1d8331a4	Merge pull request #3396 from omnivore-app/fix/queue-redis-connection ... use worker redis client when connect to the queue in queue-processor	2024-01-19 21:03:20 +08:00
Hongbo Wu	780cda6e4b	use worker redis client when connect to the queue in queue-processor	2024-01-19 20:44:54 +08:00
Hongbo Wu	338733829f	Merge pull request #3394 from omnivore-app/fix/cache-content-fetched ... fix/cache content fetched	2024-01-19 18:16:05 +08:00
Hongbo Wu	6b89f4b809	graceful shutdown redis	2024-01-19 18:06:12 +08:00
Hongbo Wu	356ad38678	reduce logs	2024-01-19 18:01:09 +08:00
Hongbo Wu	4340286b89	add retry on failure in jobs	2024-01-19 17:38:25 +08:00
Jackson Harper	b8d08ae33f	Merge pull request #3393 from omnivore-app/fix/job-priority ... Fix refresh feed job priority	2024-01-19 17:28:09 +08:00
Hongbo Wu	8acdea5002	set cache if not exists	2024-01-19 17:14:49 +08:00
Jackson Harper	defb79d8c8	Use let instead of var	2024-01-19 17:12:34 +08:00
Hongbo Wu	51d5166321	get cached content from redis when saving pages	2024-01-19 17:06:37 +08:00
Jackson Harper	8ba1242ffb	Back off refresh-all-feeds if queue is very deep	2024-01-19 16:59:41 +08:00
Jackson Harper	d2491d3eb7	Reduce refresh-all-feeds priority	2024-01-19 16:59:15 +08:00
Hongbo Wu	699d38609b	cache fetch result in redis	2024-01-19 16:58:24 +08:00
Jackson Harper	e06ff24cea	Fix logging	2024-01-19 16:44:20 +08:00
Jackson Harper	02818d94a2	Adjust priorities	2024-01-19 16:37:57 +08:00
Hongbo Wu	35805e6e8f	use user provided url for feed added	2024-01-19 16:28:37 +08:00
Jackson Harper	84375b7c36	Add validation before refreshing feed URLs	2024-01-19 16:25:05 +08:00
Jackson Harper	2dbff6a6ab	Add some more debug output	2024-01-19 16:12:44 +08:00
Jackson Harper	0e590e53a6	Improve RSS priorities for refreshing feeds	2024-01-19 16:12:22 +08:00
Hongbo Wu	d135a7fe6a	Merge pull request #3391 from omnivore-app/fix/content-fetch-dockerfile ... fix: build content-fetch in dockerfile	2024-01-19 12:45:19 +08:00
Jackson Harper	69541d3e91	Merge pull request #3392 from omnivore-app/fix/backend-svc-endpoint ... Fix construction of service endpoint	2024-01-19 12:40:30 +08:00
Hongbo Wu	3d42973c5f	fix api endpint	2024-01-19 12:36:15 +08:00
Jackson Harper	f03766d05b	Fix construction of service endpoint ... NEed to fix how we construct all these	2024-01-19 12:29:50 +08:00
Hongbo Wu	7c25a628d9	fix pem format	2024-01-19 11:44:16 +08:00
Hongbo Wu	2e8f173879	create queue on start up	2024-01-19 11:26:53 +08:00
Hongbo Wu	a6d7623cea	do not close redis connection	2024-01-19 10:39:10 +08:00
Hongbo Wu	846d1e3381	fix: build content-fetch in dockerfile	2024-01-19 10:13:05 +08:00
Hongbo Wu	57aca54538	Merge pull request #3366 from omnivore-app/feature/separate-fetch-content ... feature/separate fetch content	2024-01-18 23:30:26 +08:00
Hongbo Wu	c2a084803a	validate feed url before adding	2024-01-18 21:56:19 +08:00
Hongbo Wu	d2c17f7539	graceful shutdown redis	2024-01-18 21:52:33 +08:00
Hongbo Wu	3e291d1019	fix tests	2024-01-18 21:13:32 +08:00
Hongbo Wu	d4f49dceb6	remove job id	2024-01-18 18:48:46 +08:00
Hongbo Wu	a6f0e2f2d9	add save page job processor	2024-01-18 18:48:46 +08:00
Hongbo Wu	43df5d30a8	prioritized jobs	2024-01-18 18:48:46 +08:00
Hongbo Wu	0015a946b2	create batch save page jobs after fetching content	2024-01-18 18:48:46 +08:00
Hongbo Wu	8a3b50f07f	do not create thumbnail task if the item already has a thumbnail	2024-01-18 18:48:46 +08:00