prehistoric-systems/omnivore
1
0
Fork 0
mirror of https://github.com/omnivore-app/omnivore.git synced 2026-03-11 08:54:26 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
12057 commits 201 branches 31 tags 432 MiB
Commit graph

115 commits

Author SHA1 Message Date
Thomas Rogers
bda21730cd Remove reference to omnivore.app 2025-09-24 12:37:39 +02:00
Thomas Rogers
89e78b377d fix(): Add network timeout to all yarn installs 
Bump image-size from 1.2.0 to 1.2.1 (#4567)

Bumps [image-size](https://github.com/image-size/image-size) from 1.2.0 to 1.2.1.
- [Release notes](https://github.com/image-size/image-size/releases)
- [Commits](https://github.com/image-size/image-size/compare/v1.2.0...v1.2.1)

---
updated-dependencies:
- dependency-name: image-size
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

fix(): Add network timeout to all yarn installs

fix(): remove node-buffer

Bump brace-expansion from 1.1.11 to 1.1.12 in /pkg/admin (#4593)

Bumps [brace-expansion](https://github.com/juliangruber/brace-expansion) from 1.1.11 to 1.1.12.
- [Release notes](https://github.com/juliangruber/brace-expansion/releases)
- [Commits](https://github.com/juliangruber/brace-expansion/compare/1.1.11...v1.1.12)

---
updated-dependencies:
- dependency-name: brace-expansion
  dependency-version: 1.1.12
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bump @babel/runtime-corejs3 from 7.26.0 to 7.28.2 (#4592)

Bumps [@babel/runtime-corejs3](https://github.com/babel/babel/tree/HEAD/packages/babel-runtime-corejs3) from 7.26.0 to 7.28.2.
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.28.2/packages/babel-runtime-corejs3)

---
updated-dependencies:
- dependency-name: "@babel/runtime-corejs3"
  dependency-version: 7.28.2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bump next from 13.5.8 to 14.2.30 (#4591)

Bumps [next](https://github.com/vercel/next.js) from 13.5.8 to 14.2.30.
- [Release notes](https://github.com/vercel/next.js/releases)
- [Changelog](https://github.com/vercel/next.js/blob/canary/release.js)
- [Commits](https://github.com/vercel/next.js/compare/v13.5.8...v14.2.30)

---
updated-dependencies:
- dependency-name: next
  dependency-version: 14.2.30
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bump axios from 0.27.2 to 0.30.0 (#4590)

Bumps [axios](https://github.com/axios/axios) from 0.27.2 to 0.30.0.
- [Release notes](https://github.com/axios/axios/releases)
- [Changelog](https://github.com/axios/axios/blob/v0.30.0/CHANGELOG.md)
- [Commits](https://github.com/axios/axios/compare/v0.27.2...v0.30.0)

---
updated-dependencies:
- dependency-name: axios
  dependency-version: 0.30.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bump @babel/runtime from 7.14.6 to 7.28.2 in /pkg/admin (#4589)

Bumps [@babel/runtime](https://github.com/babel/babel/tree/HEAD/packages/babel-runtime) from 7.14.6 to 7.28.2.
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.28.2/packages/babel-runtime)

---
updated-dependencies:
- dependency-name: "@babel/runtime"
  dependency-version: 7.28.2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bump @radix-ui/react-separator from 0.1.4 to 1.1.6 (#4579)

Bumps [@radix-ui/react-separator](https://github.com/radix-ui/primitives) from 0.1.4 to 1.1.6.
- [Changelog](https://github.com/radix-ui/primitives/blob/main/release-process.md)
- [Commits](https://github.com/radix-ui/primitives/commits)

---
updated-dependencies:
- dependency-name: "@radix-ui/react-separator"
  dependency-version: 1.1.6
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bump http-proxy-middleware from 2.0.7 to 2.0.9 (#4574)

Bumps [http-proxy-middleware](https://github.com/chimurai/http-proxy-middleware) from 2.0.7 to 2.0.9.
- [Release notes](https://github.com/chimurai/http-proxy-middleware/releases)
- [Changelog](https://github.com/chimurai/http-proxy-middleware/blob/v2.0.9/CHANGELOG.md)
- [Commits](https://github.com/chimurai/http-proxy-middleware/compare/v2.0.7...v2.0.9)

---
updated-dependencies:
- dependency-name: http-proxy-middleware
  dependency-version: 2.0.9
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bump @babel/helpers from 7.14.6 to 7.28.2 in /pkg/admin (#4594)

Bumps [@babel/helpers](https://github.com/babel/babel/tree/HEAD/packages/babel-helpers) from 7.14.6 to 7.28.2.
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.28.2/packages/babel-helpers)

---
updated-dependencies:
- dependency-name: "@babel/helpers"
  dependency-version: 7.28.2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bump pg and @types/pg (#4547)

Bumps [pg](https://github.com/brianc/node-postgres/tree/HEAD/packages/pg) and [@types/pg](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/pg). These dependencies needed to be updated together.

Updates `pg` from 8.13.1 to 8.13.3
- [Changelog](https://github.com/brianc/node-postgres/blob/master/CHANGELOG.md)
- [Commits](https://github.com/brianc/node-postgres/commits/pg@8.13.3/packages/pg)

Updates `@types/pg` from 8.11.10 to 8.11.11
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/pg)

---
updated-dependencies:
- dependency-name: pg
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: "@types/pg"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bump @graphql-tools/utils from 9.2.1 to 10.8.6 (#4560)

Bumps [@graphql-tools/utils](https://github.com/ardatan/graphql-tools/tree/HEAD/packages/utils) from 9.2.1 to 10.8.6.
- [Release notes](https://github.com/ardatan/graphql-tools/releases)
- [Changelog](https://github.com/ardatan/graphql-tools/blob/master/packages/utils/CHANGELOG.md)
- [Commits](https://github.com/ardatan/graphql-tools/commits/@graphql-tools/utils@10.8.6/packages/utils)

---
updated-dependencies:
- dependency-name: "@graphql-tools/utils"
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

chore(): update versions

chore(): remove unused workload

fix(): kbar dependency

fix(): kbar dependency

chore(): fix packages

fix(): dependencies

fix(): dependencies

fix(): dependencies

fix(): dependencies api

fix(): dependencies api

fix(): dependencies api

chore(): fix order of github

fix(): fix tsconfig for liqe
 2025-07-29 19:01:37 +02:00
Tom Rogers
4e582fb55d
Improving Self-Hosting and Removing 3rd Party dependencies. (#4513)
Some checks failed
Build Docker Images / Build docker images (push) Has been cancelled
Details
Build Self-Hosting Docker Images / Build self-host docker images (push) Has been cancelled
Details
Run tests / Run Codebase tests (push) Has been cancelled
Details
Run tests / Build docker images (push) Has been cancelled
Details 
* fix: Library Header layout shift

* Bump Github Actions versions.

* Self-Hosting Changes

* Fix Minio Environment Variable

* Just make pdfs successful, due to lack of PDFHandler

* Fix issue where flag was set wrong

* Added an NGINX Example file

* Add some documentation for self-hosting via Docker Compose

* Make some adjustments to Puppeteer due to failing sites.

* adjust timings

* Add start of Mail Service

* Fix Docker Files

* More email service stuff

* Add Guide to use Zapier for Email-Importing.

* Ensure that if no env is provided it uses the old email settings

* Add some instructions for self-hosted email

* Add SNS Endpoints for Mail Watcher

* Add steps and functionality for using SES and SNS for email

* Uncomment a few jobs.

* Added option for Firefox for parser. Was having issues with Chromium on Docker.

* Add missing space.

Co-authored-by: Russ Taylor <729694+russtaylor@users.noreply.github.com>

* Fix some wording on the Guide

* update browser extension to handle self-hosted instances

* add slight documentation to options page

* Fix MV

* Do raw handlers for Medium

* Fix images in Medium

* Update self-hosting/GUIDE.md

Co-authored-by: Mike Baker <1426795+mbaker3@users.noreply.github.com>

* Update Guide with other variables

* Add The Verge to JS-less handlers

* Update regex and image-proxy

* Update self-hosting/nginx/nginx.conf

Co-authored-by: Mike Baker <1426795+mbaker3@users.noreply.github.com>

* Update regex and image-proxy

* Update self-hosting/docker-compose/docker-compose.yml

Co-authored-by: Mike Baker <1426795+mbaker3@users.noreply.github.com>

* Fix Minio for Export

* Merge to main

* Update GUIDE with newer NGINX

* Update nginx config to include api/save route

* Enable Native PDF View for PDFS

* Enable Native PDF View for PDFS

* feat:lover packages test

* feat:working build

* feat:alpine build

* docs:api dockerfile docs

* Write a PDF.js wrapper to replace pspdfkit

* Revert changes for replication, set settings to have default mode

* build folder got removed due to gitignore on pdf

* Add Box shadow to pdf pages

* Add Toggle for Progress in PDFS, enabled native viewer toggle

* Update node version to LTS

* Update node version to LTS

* Fix Linting issues

* Fix Linting issues

* Make env variable nullable

* Add touchend listener for mobile

* Make changes to PDF for mobile

* fix(android): change serverUrl to selfhosted first

* feat:2 stage alpine content fetch

* feat:separated start script

* fix:changed to node 22

* Add back youtube functionality and add guide

* trigger build

* Fix cache issue on YouTube

* Allow empty AWS_S3_ENDPOINT

* Allow empty AWS_S3_ENDPOINT

* Add GCHR for all images

* Add GCHR For self hosting.

* Add GCHR For self hosting.

* Test prebuilt.

* Test prebuilt

* Test prebuilt...

* Fix web image

* Remove Web Image (For now)

* Move docker-compose to images

* Move docker-compose files to correct locations

* Remove the need for ARGS

* Update packages, and Typescript versions

* Fix

* Fix issues with build on Web

* Correct push

* Fix Linting issues

* Fix Trace import

* Add missing types

* Fix Tasks

* Add information into guide about self-build

* Fix issues with PDF Viewer

---------

Co-authored-by: keumky2 <keumky2@woowahan.com>
Co-authored-by: William Theaker <wtheaker@nvidia.com>
Co-authored-by: Russ Taylor <729694+russtaylor@users.noreply.github.com>
Co-authored-by: David Adams <david@dadams2.com>
Co-authored-by: Mike Baker <1426795+mbaker3@users.noreply.github.com>
Co-authored-by: m1xxos <66390094+m1xxos@users.noreply.github.com>
Co-authored-by: Adil <mr.adil777@gmail.com>
 2025-01-27 13:33:16 +01:00
Hongbo Wu
70d6e9dcae reduce logs 2024-08-29 18:18:17 +08:00
Hongbo Wu
0366c426bc process up to 2 jobs concurrently 2024-08-21 17:58:58 +08:00
Hongbo Wu
03ff18af81 remove unused dependencies 2024-08-21 14:09:49 +08:00
Hongbo Wu
0fbc6d0a87 update importer to use content-fetch queue 2024-08-21 12:24:35 +08:00
Hongbo Wu
49556453e9 fix importer dependency 2024-08-13 17:38:31 +08:00
Hongbo Wu
3fe22e0f8e
Merge pull request #4215 from omnivore-app/feature/exporter 
create a cloud run service for the exporter
 2024-08-13 17:20:44 +08:00
Hongbo Wu
110f537e72 sleep for 1 second to avoid rate limiting 2024-07-22 18:36:11 +08:00
Hongbo Wu
95657e3d3e queue email sending jobs in exporter and importer 2024-07-21 11:11:29 +08:00
Hongbo Wu
01bce43781 create a cloud run service for the exporter 2024-07-19 17:12:40 +08:00
dependabot[bot]
56a97e5711
Bump @fast-csv/parse from 4.3.6 to 5.0.0 
Bumps [@fast-csv/parse](https://github.com/C2FO/fast-csv/tree/HEAD/packages/parse) from 4.3.6 to 5.0.0.
- [Release notes](https://github.com/C2FO/fast-csv/releases)
- [Changelog](https://github.com/C2FO/fast-csv/blob/main/packages/parse/CHANGELOG.md)
- [Commits](https://github.com/C2FO/fast-csv/commits/v5.0.0/packages/parse)

---
updated-dependencies:
- dependency-name: "@fast-csv/parse"
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-07-08 21:53:29 +00:00
Hongbo Wu
649463b785 only disconnect redis message queue client if it is different from cache client 2024-07-04 16:09:32 +08:00
Hongbo Wu
0a1b7345e9 add timeout 2024-07-04 12:56:28 +08:00
Hongbo Wu
bbc7b5e600 use @omnivore/utils in import-handler 2024-07-03 22:20:27 +08:00
Hongbo Wu
88a7e8d85b fix tests 2024-04-04 12:17:15 +08:00
Jackson Harper
c356183acd Remove typescript and eslint changes 2024-03-12 11:01:44 +08:00
Thomas Rogers
7d291f084b Merge all changes from main, update theming of Discover 2024-03-07 17:39:57 +01:00
Hongbo Wu
628cfc64d1 fix: replace \n with new line in redis cert 2024-01-22 11:09:15 +08:00
Hongbo Wu
8a91a3c629 fix importer tests 2024-01-18 18:48:46 +08:00
Hongbo Wu
150a456c35 replace redis client library with ioredis 2024-01-16 15:42:50 +08:00
Thomas Rogers
c0373646cb Fix Linting... 2024-01-09 11:22:32 +01:00
Hongbo Wu
32ae38fcba upgrade sentry to 7.77.0 in packages/api 2023-11-08 12:23:36 +08:00
Hongbo Wu
dd1cb188e4 fix tests 2023-10-30 14:41:53 +08:00
Hongbo Wu
47f67c237d change source to pocket if item is imported from pocket 2023-10-30 14:41:52 +08:00
Luke Channings
c4773dc904
Landing page improvements and various supporting improvements 2023-10-24 09:43:39 +01:00
Hongbo Wu
9ee87e2e52 convert type of date in the csv to int before parsing date 2023-08-15 18:44:30 +08:00
Hongbo Wu
e3654e8ae2 fix test 2023-08-14 16:38:24 +08:00
Hongbo Wu
d25f19207b add savedAt and publishedAt to the csv file 2023-08-14 13:49:06 +08:00
Hongbo Wu
ba2d34c1af validate csv file in the importer 2023-08-14 13:38:36 +08:00
Hongbo Wu
792cf0b207 upgrade google cloud sdk 2023-08-11 13:55:29 +08:00
Jackson Harper
fa1ff9ba17 Upgrade node to 18.16 2023-07-24 15:55:11 +08:00
Hongbo Wu
fc7a649566 limit import to 20k urls 2023-07-17 18:09:13 +08:00
Hongbo Wu
af7350bd31 fix: importer status update email not sent 
* create omnivore-authorization header to contain the token
 2023-06-28 11:00:40 +08:00
Hongbo Wu
8ad937ed84 allow more labels format in the csv 2023-06-14 13:33:20 +08:00
Hongbo Wu
42305ee0b4 use a separate queue for sending emails 2023-06-14 13:17:59 +08:00
Hongbo Wu
dddd763ba2 fix graphql error by setting clientRequestId empty 2023-06-13 21:15:34 +08:00
Hongbo Wu
27ff2856e2 remove client request id in import task which caused the pdf upload failure 2023-06-13 18:21:36 +08:00
Hongbo Wu
4a438f45f5 add timeout and content length to the axios requests 2023-06-12 21:31:35 +08:00
Hongbo Wu
63bddf13a1 set default metrics to zero 2023-06-05 18:16:41 +08:00
Hongbo Wu
4cad18f51c fix taskId not being set in the cloud task payload 2023-06-05 17:52:36 +08:00
Hongbo Wu
992a78e173 add dockerfile 2023-06-01 16:29:30 +08:00
Hongbo Wu
8a89a47c16 fix: testcases 2023-06-01 16:11:35 +08:00
Hongbo Wu
1073f88d69 fix: add start:collector command 2023-06-01 16:11:35 +08:00
Hongbo Wu
813a90e3c9 feat: send importer status update 2023-06-01 16:11:35 +08:00
Hongbo Wu
fba8e0424d feat: add importer metrics collector 2023-06-01 16:11:35 +08:00
Hongbo Wu
5f465112bf feat: update matter importer metrics 2023-06-01 16:11:35 +08:00
Hongbo Wu
67014e68fd feat: send email notification once the import has finished 2023-06-01 16:11:35 +08:00
Hongbo Wu
19facec4e2 feat: add importer metrics collector 2023-06-01 16:11:35 +08:00