prehistoric-systems/omnivore
1
0
Fork 0
mirror of https://github.com/omnivore-app/omnivore.git synced 2026-03-11 08:54:26 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
1617 commits 201 branches 31 tags 432 MiB
Commit graph

259 commits

Author SHA1 Message Date
Jackson Harper
b0fe9059a9 Dont try to generate highlight URL previews until share is re-enabled 2022-03-09 09:49:03 -08:00
Jackson Harper
a5a42a36e7
Merge pull request #206 from omnivore-app/dependabot/npm_and_yarn/opentelemetry/exporter-jaeger-1.0.1 
Bump @opentelemetry/exporter-jaeger from 0.24.0 to 1.0.1
 2022-03-08 21:00:53 -08:00
dependabot[bot]
9bda6ede06
Bump @opentelemetry/exporter-jaeger from 0.24.0 to 1.0.1 
Bumps [@opentelemetry/exporter-jaeger](https://github.com/open-telemetry/opentelemetry-js) from 0.24.0 to 1.0.1.
- [Release notes](https://github.com/open-telemetry/opentelemetry-js/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-js/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-js/compare/v0.24.0...stable/v1.0.1)

---
updated-dependencies:
- dependency-name: "@opentelemetry/exporter-jaeger"
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-03-09 04:45:18 +00:00
Jackson Harper
384a0771a7
Merge pull request #200 from omnivore-app/fix/validate-urls 
Use the validateUrl method to validate URLs
 2022-03-08 20:28:33 -08:00
Jackson Harper
c45c408c14 Fix formatting 2022-03-08 15:27:05 -08:00
Jackson Harper
e8fca4a7a9 Remove debug line 2022-03-08 15:20:24 -08:00
Jackson Harper
354fe36dd5 Add a test for save email 2022-03-08 15:17:11 -08:00
Jackson Harper
26dadab4aa rm debug, we dont need to set claims on create 2022-03-08 15:10:01 -08:00
Jackson Harper
b982bf34d6 SetClaims on userArticle create/update in saveEmail 2022-03-08 14:43:10 -08:00
Jackson Harper
9ae81d7394 Add extra debugging on newsletter save errors 2022-03-08 14:00:35 -08:00
Jackson Harper
2184c2a8d3 Parse online URLs for beehiiv newsletters 2022-03-07 15:49:44 -08:00
Jackson Harper
2cb5cc065a Add support for identifying newsletters hosted on beehiiv.com 2022-03-07 15:23:56 -08:00
Jackson Harper
bba094fbcd Add tests for beehiiv.com hosted newsletters 2022-03-07 15:05:38 -08:00
Jackson Harper
be7e36ffed Use the validateUrl method to validate URLs 2022-03-07 14:34:32 -08:00
Jackson Harper
ae329d2651 Add test data file 2022-03-03 19:47:37 -08:00
Jackson Harper
b6fd3e786e Fix parsing authors from page metadata 2022-03-03 19:40:02 -08:00
Jackson Harper
49092b707d Remove async 2022-03-03 19:31:51 -08:00
Jackson Harper
05373ba3c7 add methid to parse content metadata 2022-03-03 17:10:06 -08:00
Jackson Harper
c2e08d0e8f Fetch title and author from page metadate if possible 2022-03-03 15:20:58 -08:00
Jackson Harper
b17f22949c Use the readability title if available 2022-03-03 14:21:30 -08:00
Jackson Harper
a4bb67deee Use the readability byline for email authors if available 2022-03-03 14:21:03 -08:00
Jackson Harper
b326a5f8e7 Add more matches on substack icons 2022-03-03 13:42:30 -08:00
Jackson Harper
484cd78ac5 prettier 2022-03-02 23:14:10 -08:00
Jackson Harper
65ce8353dc Attempt to pull URLs for probable newsletter emails out of content 2022-03-02 23:09:10 -08:00
Jackson Harper
21329949e5 Fix to email address in saveNewsletterEmail 2022-03-02 22:06:15 -08:00
Jackson Harper
02d505a7e2 Add some debugging 2022-03-02 21:39:10 -08:00
Jackson Harper
8fe9827ad2 If an email appears to be a newsletter at it to the library 2022-03-02 20:47:53 -08:00
Jackson Harper
65cc666579 Pass HTML instead of a JSDOM into isProbablyNewsletter to better encapsulate 2022-03-02 20:38:11 -08:00
Jackson Harper
c4e237927d Allow any on GCP func 2022-03-02 20:27:40 -08:00
Jackson Harper
e660e41ed5 Fix warnings 2022-03-02 19:49:10 -08:00
Jackson Harper
9206230659 Better name for the save newsletter service 2022-03-02 19:45:28 -08:00
Jackson Harper
d66c114a7d Remove file, this is moved into the parser 2022-03-02 16:36:36 -08:00
Jackson Harper
b5f9478350 Dont mutate function input 2022-03-02 16:34:56 -08:00
Jackson Harper
f7f83fe080 New function to determine if an HTML blob is probably a newsletter based on its content 2022-03-02 16:31:15 -08:00
dependabot[bot]
29c941f4ae
Bump apollo-datasource from 0.7.3 to 3.3.1 
Bumps [apollo-datasource](https://github.com/apollographql/apollo-server/tree/HEAD/packages/apollo-datasource) from 0.7.3 to 3.3.1.
- [Release notes](https://github.com/apollographql/apollo-server/releases)
- [Changelog](https://github.com/apollographql/apollo-server/blob/main/CHANGELOG.md)
- [Commits](https://github.com/apollographql/apollo-server/commits/apollo-datasource@3.3.1/packages/apollo-datasource)

---
updated-dependencies:
- dependency-name: apollo-datasource
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-03-01 23:19:04 +00:00
Jackson Harper
505f888e37 Update to latest intercom client 2022-03-01 11:20:31 -08:00
dependabot[bot]
cadccfc7e1
Bump intercom-client from 2.11.2 to 3.1.4 
Bumps [intercom-client](https://github.com/intercom/intercom-node) from 2.11.2 to 3.1.4.
- [Release notes](https://github.com/intercom/intercom-node/releases)
- [Commits](https://github.com/intercom/intercom-node/commits/v3.1.4)

---
updated-dependencies:
- dependency-name: intercom-client
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-03-01 18:40:33 +00:00
Hongbo Wu
fc9aa9452c
Add a flag in readability to retain table elements in newsletter emails (#152) 
* add a flag in readability to retain table elements in newsletter emails

* remove header of axios newsletters
 2022-03-01 11:49:38 +08:00
dependabot[bot]
b2faf8a78d
Bump @opentelemetry/instrumentation-express from 0.24.0 to 0.28.0 
Bumps [@opentelemetry/instrumentation-express](https://github.com/open-telemetry/opentelemetry-js-contrib) from 0.24.0 to 0.28.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-js-contrib/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-js-contrib/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-js-contrib/compare/v0.24.0...contrib-test-utils-v0.28.0)

---
updated-dependencies:
- dependency-name: "@opentelemetry/instrumentation-express"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-02-28 21:27:15 +00:00
Hongbo Wu
7bf454ae91
use dataloader to fetch all labels of a list of linkIds in a single q… (#133) 
* use dataloader to fetch all labels of a list of linkIds in a single query and cached

* add labels in GQL query in frontend
 2022-02-28 12:13:26 +08:00
Jackson Harper
328ebc48cb Apply code block highlighting before running DOM clean 2022-02-26 19:04:32 -08:00
Jackson Harper
e0b8cfc938 Import highlightjs types 
We still need these dev packages because of the way our docker
builds are setup.
 2022-02-26 17:37:01 -08:00
Jackson Harper
fd39923907 Prettier improvements 2022-02-26 16:38:43 -08:00
Jackson Harper
84fbc9cd27 Add code highlighting using highlight.js 2022-02-26 14:57:59 -08:00
Satindar Dhillon
e27c2bdd8b generate swift gql code 2022-02-25 10:09:17 -08:00
Hongbo Wu
b9d581166e fix merge conflicts 2022-02-24 14:56:00 +08:00
Hongbo Wu
42f2cffdf8 remove debug log 2022-02-24 14:47:33 +08:00
Hongbo Wu
276205d52a add linkId and labels in article type 2022-02-24 14:44:49 +08:00
Hongbo Wu
c7841b8e8a remove duplicate links because of join with link_labels table 2022-02-24 14:44:49 +08:00
Hongbo Wu
fc18004a5d add labels in search query 2022-02-24 14:44:49 +08:00
Hongbo Wu
edb9339897 add labels and link_labels table name 2022-02-24 14:44:49 +08:00
Hongbo Wu
5651c92efe ingore case when checking name uniqueness 2022-02-24 14:44:49 +08:00
Hongbo Wu
9599c98c77 add label filters in search query 2022-02-24 14:44:49 +08:00
Hongbo Wu
fc37d06fd2 update params variable 2022-02-24 14:44:49 +08:00
Hongbo Wu
d33b213092 add setLabels api and tests 2022-02-24 14:44:48 +08:00
Hongbo Wu
9150e7dbc2 add tests for create/delete labels 2022-02-24 14:44:48 +08:00
Hongbo Wu
e74f334db9 update entity 2022-02-24 14:44:48 +08:00
Jackson Harper
6e6745a2b7 Use version 6.1.0 for normalize-url which switches to modules in v7 2022-02-23 10:28:28 -08:00
dependabot[bot]
385cfdad71
Bump normalize-url from 5.3.1 to 7.0.3 
Bumps [normalize-url](https://github.com/sindresorhus/normalize-url) from 5.3.1 to 7.0.3.
- [Release notes](https://github.com/sindresorhus/normalize-url/releases)
- [Commits](https://github.com/sindresorhus/normalize-url/commits/v7.0.3)

---
updated-dependencies:
- dependency-name: normalize-url
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-02-23 18:14:46 +00:00
Jackson Harper
7c9c75836b Update the monitor packages as these are peer dependencies 2022-02-23 09:26:48 -08:00
Jackson Harper
5e2bba3231 Upgrade opentelemetry packages 2022-02-22 12:42:19 -08:00
dependabot[bot]
4a9d24025b Bump @opentelemetry/node from 0.22.0 to 0.24.0 
Bumps [@opentelemetry/node](https://github.com/open-telemetry/opentelemetry-js) from 0.22.0 to 0.24.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-js/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-js/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-js/compare/v0.22.0...v0.24.0)

---
updated-dependencies:
- dependency-name: "@opentelemetry/node"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-02-22 12:42:19 -08:00
Jackson Harper
5f6f3ba3b2 Fix warnings in the API package 2022-02-21 16:46:10 -08:00
Jackson Harper
a276af5264 Remove extra white space 2022-02-21 13:55:09 -08:00
Jackson Harper
cf6733c70f When fetching the library item list dont add content/originalHtml from DB 2022-02-21 13:55:09 -08:00
Jackson Harper
5af4899415 Use null instead of undefined when unarchiving links 
This fixes unarchiving content by re-saving it via the Save*
API methods.

Setting to undefined is not the same as null in this case, and
will result in archived_at being non-null and the link then
being excluded from the GetArticles query.
 2022-02-21 11:26:22 -08:00
Hongbo Wu
4c84c40f97
add slug to the link (#84) 2022-02-18 10:43:34 +08:00
Jackson Harper
05aa5d737e
Merge pull request #82 from omnivore-app/feature/bloomberg-newsletters 
Add support for Bloomberg newsletter
 2022-02-18 01:08:46 +08:00
Hongbo Wu
9431e80be8 add parser 2022-02-17 20:17:02 +08:00
Hongbo Wu
e3839647e1 parse bloomberg newsletter 2022-02-17 20:14:06 +08:00
Hongbo Wu
ad8bca3d84 use subject as title 2022-02-17 18:59:07 +08:00
Hongbo Wu
7229c64da0
fix: Should not get fake uploading url in demo (#79) 
* fix not getting fake uploading url in demo

* use internal service endpoint env

* update internal svc endpoint
 2022-02-17 13:34:50 +08:00
Hongbo Wu
bd8178a464 add support for bloomberg newsletters 2022-02-17 13:19:58 +08:00
Jackson Harper
3a27681534
Merge pull request #74 from omnivore-app/chore/upgrade-gql-typegen-2.6.2 
Upgrade gql-codegen, regenerate GQL schema
 2022-02-17 04:03:31 +08:00
Jackson Harper
e80eb7ab06 Upgrade gql-codegen, regenerate GQL schema 2022-02-16 11:32:13 -08:00
Hongbo Wu
c1ffd9abdf fix sanitize bug by having different SanitizedString type name when creating new instances 2022-02-16 09:33:33 -08:00
Jackson Harper
e327b7c57f Expect a 400 error for failed GQL directives 2022-02-16 09:33:33 -08:00
Jackson Harper
bf305bef39 Make sanitize test more explicit 2022-02-16 09:33:33 -08:00
Jackson Harper
c8abf87367 Cast the directive types 2022-02-16 09:33:33 -08:00
Jackson Harper
6d5554dd5f Implement the sanitize directive in applo 3.0 2022-02-16 09:33:33 -08:00
Jackson Harper
7e35281bd1 createApp is no longer async, linting 2022-02-16 09:33:33 -08:00
Jackson Harper
fb224d60c4 Remove subscriptions, comment out the sanitize directive 2022-02-16 09:33:30 -08:00
Jackson Harper
df2b36b33a Upgrade apollo-server-express, remove connections/subscriptions 2022-02-16 09:32:48 -08:00
Jackson Harper
5a457479e2
Merge pull request #64 from omnivore-app/feature/handle-pdf-attachment 
add pdf attachment to library
 2022-02-17 00:55:27 +08:00
Hongbo Wu
d53e94aeb2 add tests for create article 2022-02-16 16:14:14 +08:00
Hongbo Wu
77ba7531f3 add tests for upload 2022-02-16 15:26:11 +08:00
Hongbo Wu
9460197132 add more logs for debugging 2022-02-16 11:34:01 +08:00
Hongbo Wu
b826a101c5 add pdf attachment create-article 2022-02-16 09:24:16 +08:00
Jackson Harper
340d7903b3
Merge pull request #67 from omnivore-app/fix/upgrade-gql-typegen 
Upgrade gql-typegen and regen types
 2022-02-16 02:52:02 +08:00
Jackson Harper
c3308085ee Upgrade gql-typegen and regen types 2022-02-15 10:42:05 -08:00
Jackson Harper
06a38ae407
Merge pull request #55 from omnivore-app/dependabot/npm_and_yarn/opentelemetry/instrumentation-grpc-0.27.0 
Bump @opentelemetry/instrumentation-grpc from 0.22.0 to 0.27.0
 2022-02-16 02:05:58 +08:00
Hongbo Wu
5d95e6c734 add pdf attachment upload 2022-02-15 22:22:13 +08:00
Jackson Harper
49e436b10c Remove unneeded installs from API docker file 
These were needed when Chromium was installed on this image.
 2022-02-14 21:52:04 -08:00
dependabot[bot]
f93a3e3070
Bump @opentelemetry/instrumentation-grpc from 0.22.0 to 0.27.0 
Bumps [@opentelemetry/instrumentation-grpc](https://github.com/open-telemetry/opentelemetry-js) from 0.22.0 to 0.27.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-js/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-js/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-js/compare/v0.22.0...experimental/v0.27.0)

---
updated-dependencies:
- dependency-name: "@opentelemetry/instrumentation-grpc"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-02-15 05:22:49 +00:00
Hongbo Wu
e85273d87a
add support for axios newsletters (#49) 
* add support for axios newsletters

* fix raw url

* fix getting newsletter url from axios

* add test for url

* add axios parser
 2022-02-15 11:56:03 +08:00
Jackson Harper
65cd8a95eb Better handling of situations where we fail to set the cookie during auth 2022-02-14 16:04:23 -08:00
Jackson Harper
165085a639 Handle upgraded axios API for headers 2022-02-14 15:48:36 -08:00
Jackson Harper
238a2f0e97
Merge pull request #56 from omnivore-app/dependabot/npm_and_yarn/axios-0.26.0 
Bump axios from 0.21.4 to 0.26.0
 2022-02-15 07:25:37 +08:00
Jackson Harper
a23ae33264
Merge pull request #54 from omnivore-app/dependabot/npm_and_yarn/opentelemetry/instrumentation-dns-0.27.1 
Bump @opentelemetry/instrumentation-dns from 0.22.0 to 0.27.1
 2022-02-15 07:24:16 +08:00
Jackson Harper
952e110ccd Remove unneeded values from api env example file 2022-02-14 14:56:07 -08:00
dependabot[bot]
0f843809e3
Bump axios from 0.21.4 to 0.26.0 
Bumps [axios](https://github.com/axios/axios) from 0.21.4 to 0.26.0.
- [Release notes](https://github.com/axios/axios/releases)
- [Changelog](https://github.com/axios/axios/blob/master/CHANGELOG.md)
- [Commits](https://github.com/axios/axios/compare/v0.21.4...v0.26.0)

---
updated-dependencies:
- dependency-name: axios
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-02-14 21:35:58 +00:00
dependabot[bot]
77598cd093
Bump @opentelemetry/instrumentation-dns from 0.22.0 to 0.27.1 
Bumps [@opentelemetry/instrumentation-dns](https://github.com/open-telemetry/opentelemetry-js-contrib) from 0.22.0 to 0.27.1.
- [Release notes](https://github.com/open-telemetry/opentelemetry-js-contrib/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-js-contrib/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-js-contrib/compare/v0.22.0...host-metrics-v0.27.1)

---
updated-dependencies:
- dependency-name: "@opentelemetry/instrumentation-dns"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-02-14 21:21:48 +00:00
Hongbo Wu
a04e91472d
fix a bug of using article as table name in the reminders query (#34) 
* fix a bug of using article as table name in the reminders query

* fix test

* increase timeout value

* increase timeout value to 50000

* add import mocha

* add import mocha for label test

* add import mocha for every test

* fix test in deleting labels

* add timeout for each label test

* one more test

* fix reminders router by allowing sendrid template id to be null

* do not retry if reminders not found
 2022-02-14 13:59:04 +08:00
dependabot[bot]
c10cd4d831
Bump @opentelemetry/instrumentation-http from 0.22.0 to 0.27.0 
Bumps [@opentelemetry/instrumentation-http](https://github.com/open-telemetry/opentelemetry-js) from 0.22.0 to 0.27.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-js/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-js/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-js/compare/v0.22.0...experimental/v0.27.0)

---
updated-dependencies:
- dependency-name: "@opentelemetry/instrumentation-http"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-02-12 21:32:38 +00:00
Jackson Harper
28a208ba2c Remove some unneeded logging on startup 2022-02-12 09:16:45 -08:00
Jackson Harper
8163b5a4de Upgrade axios 2022-02-11 13:51:43 -08:00
Jackson Harper
ef2b53e7f7 Upgrade Cloud Storage and the apollo-express libraries 2022-02-11 13:21:03 -08:00
Jackson Harper
ca7b79d1ba Simplify docker compose setup for new users 
This takes out some environment variables that are not needed
for new users testing out locally.

We also adjusted the port usage so docker-compose will use the
same ports that local dev typically uses.
 2022-02-11 11:45:37 -08:00
Jackson Harper
84f32935f5 Open source omnivore 2022-02-11 09:24:33 -08:00