Commit graph

10513 commits

Author SHA1 Message Date
Hongbo Wu
35805e6e8f use user provided url for feed added 2024-01-19 16:28:37 +08:00
Jackson Harper
84375b7c36 Add validation before refreshing feed URLs 2024-01-19 16:25:05 +08:00
Jackson Harper
2dbff6a6ab Add some more debug output 2024-01-19 16:12:44 +08:00
Jackson Harper
0e590e53a6 Improve RSS priorities for refreshing feeds 2024-01-19 16:12:22 +08:00
Hongbo Wu
d135a7fe6a
Merge pull request #3391 from omnivore-app/fix/content-fetch-dockerfile
fix: build content-fetch in dockerfile
2024-01-19 12:45:19 +08:00
Jackson Harper
69541d3e91
Merge pull request #3392 from omnivore-app/fix/backend-svc-endpoint
Fix construction of service endpoint
2024-01-19 12:40:30 +08:00
Hongbo Wu
3d42973c5f fix api endpint 2024-01-19 12:36:15 +08:00
Jackson Harper
f03766d05b Fix construction of service endpoint
NEed to fix how we construct all these
2024-01-19 12:29:50 +08:00
Hongbo Wu
7c25a628d9 fix pem format 2024-01-19 11:44:16 +08:00
Hongbo Wu
2e8f173879 create queue on start up 2024-01-19 11:26:53 +08:00
Hongbo Wu
a6d7623cea do not close redis connection 2024-01-19 10:39:10 +08:00
Hongbo Wu
846d1e3381 fix: build content-fetch in dockerfile 2024-01-19 10:13:05 +08:00
Hongbo Wu
57aca54538
Merge pull request #3366 from omnivore-app/feature/separate-fetch-content
feature/separate fetch content
2024-01-18 23:30:26 +08:00
Hongbo Wu
c2a084803a validate feed url before adding 2024-01-18 21:56:19 +08:00
Hongbo Wu
d2c17f7539 graceful shutdown redis 2024-01-18 21:52:33 +08:00
Hongbo Wu
3e291d1019 fix tests 2024-01-18 21:13:32 +08:00
Hongbo Wu
d4f49dceb6 remove job id 2024-01-18 18:48:46 +08:00
Hongbo Wu
a6f0e2f2d9 add save page job processor 2024-01-18 18:48:46 +08:00
Hongbo Wu
43df5d30a8 prioritized jobs 2024-01-18 18:48:46 +08:00
Hongbo Wu
0015a946b2 create batch save page jobs after fetching content 2024-01-18 18:48:46 +08:00
Hongbo Wu
8a3b50f07f do not create thumbnail task if the item already has a thumbnail 2024-01-18 18:48:46 +08:00
Hongbo Wu
5136874372 remove debug logs 2024-01-18 18:48:46 +08:00
Hongbo Wu
a03eee5ef7 fix dependecies 2024-01-18 18:48:46 +08:00
Hongbo Wu
d9feb740cb convert content-fetch to typescript 2024-01-18 18:48:46 +08:00
Hongbo Wu
cd3402b98a rewrite puppeteer in typescript 2024-01-18 18:48:46 +08:00
Hongbo Wu
51e586ed3d separate content-fetch in puppeteer packages from saving page content 2024-01-18 18:48:46 +08:00
Hongbo Wu
c297a8dd4e feat: group fetch content tasks by url 2024-01-18 18:48:46 +08:00
Hongbo Wu
8a91a3c629 fix importer tests 2024-01-18 18:48:46 +08:00
Jackson Harper
3f163a99c7
Merge pull request #3389 from omnivore-app/fix/refresh-feed-api-url
Fix endpoint for internal refresh feed API call
2024-01-18 17:13:05 +08:00
Jackson Harper
3f707c06f2 Fix checks for env vars 2024-01-18 16:59:53 +08:00
Jackson Harper
3b0338967d Fix endpoint for internal refresh feed API call
This should eventually be replaced.
2024-01-18 16:48:45 +08:00
Jackson Harper
bffe31bcc7
Merge pull request #3359 from omnivore-app/feat/backend-bull-job-processor
Use bullmq to process jobs
2024-01-18 15:38:30 +08:00
Jackson Harper
6193c78288 Fix typo 2024-01-18 15:37:38 +08:00
Jackson Harper
7754139207 Remove debug 2024-01-18 15:33:51 +08:00
Jackson Harper
881a98eb86 Create new environment variables for tts redis 2024-01-18 14:57:35 +08:00
Jackson Harper
fb28d7b3d1 Enqueue new feeds at front 2024-01-18 14:34:07 +08:00
Jackson Harper
ee142d3d7c Handle SIGINT in server better by moving DB call to after redis shutdown 2024-01-18 12:00:13 +08:00
Jackson Harper
bb55137bb4 Disconnect one client while quitting the other 2024-01-18 11:39:41 +08:00
Jackson Harper
c800114045
Merge pull request #3385 from se-jaeger/main
Helm Chart for self-hosting
2024-01-18 11:29:50 +08:00
Jackson Harper
fa685d4a29 Tear down redis with disconnect instead of quit 2024-01-18 11:28:09 +08:00
Hongbo Wu
d558fe2b17 fix tests 2024-01-18 11:24:26 +08:00
Jackson Harper
5a7056b7ec Dont fail whole job if we cant update a subscription group 2024-01-18 10:18:45 +08:00
Sebastian Jäger
97c1db6e42
minor improvements 2024-01-17 17:16:49 +01:00
Sebastian Jäger
1d4ebb5132
minor changes and docs 2024-01-17 17:15:56 +01:00
Sebastian Jäger
bce26b9e54
Merge branch 'omnivore-app:main' into main 2024-01-17 15:35:22 +01:00
Jackson Harper
65347a1996 Use quit when shutting down redis 2024-01-17 17:30:16 +08:00
Jackson Harper
fd81f65e53 Remove server.close call 2024-01-17 16:37:51 +08:00
Jackson Harper
9e6b749604 Update error message 2024-01-17 16:21:42 +08:00
Jackson Harper
e1e2f86219 Remove duplicated error 2024-01-17 16:20:42 +08:00
Jackson Harper
dc4126d52c REvert some unneeded env changes 2024-01-17 16:15:00 +08:00