prehistoric-systems/omnivore
1
0
Fork 0
mirror of https://github.com/omnivore-app/omnivore.git synced 2026-03-11 08:54:26 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
1617 commits 201 branches 31 tags 432 MiB
Commit graph

259 commits

Author SHA1 Message Date
Hongbo Wu
fd18affaec fix document_missing_exception being thrown to sentry when updating page in elastic 2022-03-28 21:16:33 +08:00
Hongbo Wu
b6c04f3508 fix saving report item by linking to elastic page 2022-03-28 20:38:54 +08:00
Jackson Harper
1cdbfaca26
Merge pull request #294 from omnivore-app/feature/add-created-or-updated_at-to-newsletter_email 
add created_at and updated_at to newsletter emails and sort by create…
 2022-03-25 09:10:54 -07:00
Jackson Harper
5036924b21
Merge pull request #280 from omnivore-app/dependabot/npm_and_yarn/jsdom-19.0.0 
Bump jsdom from 16.7.0 to 19.0.0
 2022-03-25 09:08:55 -07:00
Hongbo Wu
27735f7310
add page id in updated page json file in gcs (#319) 2022-03-25 13:50:43 +08:00
Hongbo Wu
d595d4a8c4
add success and failure count in assertions (#318) 2022-03-25 12:16:43 +08:00
Hongbo Wu
e7203bebb5
add a function_resolver to return originalArticleUrl from article.url (#317) 
* add a function_resolver to return originalArticleUrl from article.url

* add a test
 2022-03-25 11:25:30 +08:00
Jackson Harper
0264bc6733
Merge pull request #283 from omnivore-app/dependabot/npm_and_yarn/sentry/integrations-6.19.1 
Bump @sentry/integrations from 5.30.0 to 6.19.1
 2022-03-23 20:05:09 -07:00
dependabot[bot]
3ec6f22199
Bump jsdom from 16.7.0 to 19.0.0 
Bumps [jsdom](https://github.com/jsdom/jsdom) from 16.7.0 to 19.0.0.
- [Release notes](https://github.com/jsdom/jsdom/releases)
- [Changelog](https://github.com/jsdom/jsdom/blob/master/Changelog.md)
- [Commits](https://github.com/jsdom/jsdom/compare/16.7.0...19.0.0)

---
updated-dependencies:
- dependency-name: jsdom
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-03-23 18:51:08 +00:00
Hongbo Wu
79a7eb11a3 fix data loss by selecting data from links instead of pages 2022-03-23 17:27:55 +08:00
Hongbo Wu
8a485af20d validate publishedAt before inserting to elastic 2022-03-23 14:30:33 +08:00
Hongbo Wu
e1752bb176 uncomment testing code 2022-03-23 12:37:33 +08:00
Hongbo Wu
1b0399f286 add assertion of migrated data in python script 2022-03-23 12:09:25 +08:00
Hongbo Wu
6597cc37b6 add created_at and updated_at to newsletter emails and sort by created_at desc 2022-03-22 18:43:47 +08:00
Hongbo Wu
acec3b1acb Use lower priority queue for large imports 
* add a count query in elastic
  * if priority is not set, check the rate limit of user's libary
    - if 5 articles added in the last minute: use low queue
    - default: use normal queue
 2022-03-22 18:08:08 +08:00
Hongbo Wu
7d19c933ac add elastic in README 2022-03-22 15:33:51 +08:00
Hongbo Wu
1893e36375
Fix uploading endpoint returns 504 (#289) 
* add timeout = 60s for uploading to private bucket

* add debug logs

* send response to client to close connection

* add tests for upload

* add PUBSUB_VERIFICATION_TOKEN in .env.test
 2022-03-22 14:20:40 +08:00
Hongbo Wu
3e9063c145
return empty content instead of null in search result (#291) 2022-03-22 13:34:44 +08:00
Hongbo Wu
bee3b3c6fa
fix article_id type mismatch (uuid => varchar) when saving highlights in postgres by setting article_id to be undefined (#285) 2022-03-22 11:48:01 +08:00
Hongbo Wu
4ca7c89622
format date string as yyyy-mm-dd (#288) 2022-03-22 11:46:39 +08:00
Hongbo Wu
c15efe6aad
Fix sentry log size error (#287) 
* reduce sentry log size error by partially logging page created in elastic

* remove some debugging logs

* remove more debugging logs
 2022-03-22 11:45:58 +08:00
dependabot[bot]
783118a2e4
Bump @sentry/integrations from 5.30.0 to 6.19.1 
Bumps [@sentry/integrations](https://github.com/getsentry/sentry-javascript) from 5.30.0 to 6.19.1.
- [Release notes](https://github.com/getsentry/sentry-javascript/releases)
- [Changelog](https://github.com/getsentry/sentry-javascript/blob/master/CHANGELOG.md)
- [Commits](https://github.com/getsentry/sentry-javascript/compare/5.30.0...6.19.1)

---
updated-dependencies:
- dependency-name: "@sentry/integrations"
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-03-21 21:19:05 +00:00
Jackson Harper
f9bce1932c Print the logging level on startup 2022-03-21 09:31:05 -07:00
Hongbo Wu
77050905a5 retry max three times if update labels is conflicted 2022-03-18 10:39:11 +08:00
Hongbo Wu
9d8d2e40d7 reduce time to save reading progress in elastic update 2022-03-18 10:29:01 +08:00
Hongbo Wu
ac567f7e71 retry max three times if update is conflicted 2022-03-18 10:17:20 +08:00
Jackson Harper
7910565dbf
Merge pull request #251 from omnivore-app/fix/jsonld-encoding 
Use html decoding when getting values from fetched oembed
 2022-03-17 16:53:21 -07:00
Hongbo Wu
563131ea23
Fix/newsletter label not add (#256) 
* default sort by savedAt to avoid recent read page to appear on top of the list

* revert lint on generated code

* fix elastic update script syntax error
 2022-03-17 13:22:38 +08:00
Hongbo Wu
a2ce98229e
default sort by savedAt to avoid recent read page to appear on top of… (#254) 
* default sort by savedAt to avoid recent read page to appear on top of the list

* revert lint on generated code
 2022-03-17 10:54:00 +08:00
Hongbo Wu
af037a2837
make readingProgress required in the elastic page data (#253) 
* make readingProgress required in the elastic page data

* delete readingProgress from function_resolvers because we have stored them in elastic
 2022-03-17 10:06:21 +08:00
Jackson Harper
ff1200f3a1 Use html decoding when getting values from fetched oembed 
If we fetch oembed data from an external source, instead of
handling it in readabilityjs we need to html decode it.
 2022-03-16 15:29:42 -07:00
Jackson Harper
89f3719ba3 Fix linting 2022-03-16 14:23:39 -07:00
Jackson Harper
0314d8cf17 Add some extra logging 2022-03-16 13:53:55 -07:00
Jackson Harper
dc27c0c7a4 Return 0 if looking up page reading progress fails 2022-03-16 13:33:36 -07:00
Jackson Harper
27143cbedf Remove function handlers for reading progress 
These shouldn't be needed anymore because the values are stored
in elastic.
 2022-03-16 13:11:36 -07:00
Jackson Harper
88103cba9f
Merge pull request #162 from omnivore-app/dependabot/npm_and_yarn/apollo-datasource-3.3.1 
Bump apollo-datasource from 0.7.3 to 3.3.1
 2022-03-16 11:03:26 -07:00
Hongbo Wu
07dcd5da26 add label to page only if not exists 2022-03-16 23:00:37 +08:00
Hongbo Wu
1c4dcd7b00 automatically update updatedAt when page is updated in elastic 2022-03-16 22:28:39 +08:00
Hongbo Wu
f412758040 add Newsletter label to the page created by newsletters email 2022-03-16 18:40:05 +08:00
Hongbo Wu
27157006c1
use private bucket to upload page events (#244) 
* use private bucket to upload page events

* fix tests

* add GCS_UPLOAD_PRIVATE_BUCKET in test env

* allow GCS_UPLOAD_PRIVATE_BUCKET to be empty
 2022-03-16 14:39:07 +08:00
Jackson Harper
e652a6ea8c
Rebased version of the elastic PR (#225) 
* Add elastic to our docker compose

* add AND/OR/NOT search operations

* add elastic and create article in elastic

* change error code when elastic throws error

* add search pages in elastic

* add search by labels

* Add elastic to GitHub Action

* Update elastic version

* Fix port for elastic

* add url in search query

* Set elastic features when running tests

* add debug logs

* Use localhost instead of service hostname

* refresh elastic after create/update

* update search labels query

* add typescript support

* search pages in elastic

* fix search queries

* use elastic for saving page

* fix test failure

* update getArticle api to use elastic

* use generic get page function

* add elastic migration python script

* fix bulk helper param

* save elastic page id in article_saving_request instead of postgres article_id

* fix page archiving and deleting

* add tests for deleteArticle

* remove custom date type in elastic mappings which not exist in older version of elastic

* fix timestamp format issue

* add tests for save reading progress

* add tests for save file

* optimize search results

* add alias to index

* update migration script to receive env var as params

* Add failing test to validate we don't decrease reading progress

This test is failing with Elastic because we aren't fetching
the reading progress from elastic here, and are fetching it
from postgres.

* Rename readingProgress to readingProgressPercent

This is the name stored in elastic, so fixes issues pulling the
value out.

* Linting

* Add failing test for creating highlights w/elastic

This test fails because the highlight can't be looked up. Is there
a different ID we should be passing in to query for highlights,
or do we need to update the query to look for elastic_id?

* add tests code coverage threshold

* update nyc config

* include more files in test coverage

* change alias name

* update updateContent to update pages in elastic

* remove debug log

* fix createhighlight test

* search pages by alias in elastic

* update set labels and delete labels in elastic

* migration script enumeration

* make BULK_SIZE an env var

* fix pdf search indexing

* debug github action exit issue

* call pubsub when create/update/delete page in elastic

* fix json parsing bug and reduce reading data from file

* replace a depreciated pubsub api call

* debug github action exit issue

* debug github action exit issue

* add handler to upload elastic page data to GCS

* fix tests

* Use http_auth instead of basic_auth

* add index creation and existing postgres tables update in migration script

* fix a typo to connect to elastic

* rename readingProgress to readingProgressPercent

* migrate elastic_page_id in highlights and article_saving_request tables

* update migration script to include number of updated rows

* update db migration query

* read index mappings from file

* fix upload pages to gcs

* fix tests failure due to pageContext

* fix upload file id not exist error

* Handle savedAt & isArchived attributes w/out quering elastic

* Fix prettier issues

* fix content-type mismatching

* revert pageId to linkId because frontend was not deployed yet

* fix newsletters and attachment not saved in elastic

* put linkId in article for setting labels

* exclude orginalHtml in the result of searching to improve performace

* exclude content in the result of searching to improve performace

* remove score sorting

* do not refresh immediately to reduce searching and indexing time

* do not replace the backup data in gcs

* fix no article id defined in articleSavingRequest

* add logging of elastic api running time

* reduce home feed pagination size to 15

* reduce home feed pagination size to 10

* stop revalidating first page

* do not use a separate api to fetch reading progress

* Remove unused comment

* get reading progress if not exists

* replace ngram tokenizer with standard tokenizer

* fix tests

* remove .env.local

* add sort keyword in searching to sort by score

Co-authored-by: Hongbo Wu <hongbo@omnivore.app>
 2022-03-16 12:08:59 +08:00
Jackson Harper
a4533dc016
Merge pull request #201 from omnivore-app/feature/beehiiv-newsletter-support 
Support newsletters hosted on beehiiv
 2022-03-15 14:03:55 -07:00
Jackson Harper
8e1b4fb1a4 Formatting 2022-03-14 15:36:17 -07:00
Jackson Harper
a81181ee60 Dont make queries for readingProgressPercent unless we have to 2022-03-14 15:22:53 -07:00
Jackson Harper
fc7d972855 Fix typo in readability date handling causing this parse issue 
Can remove our special handler for the published date now that we
are pulling it out correctly.
 2022-03-14 10:20:19 -07:00
Jackson Harper
5983cfe2a6 Attempt to set publishedDate if readability fails to parse it 
This can happen if JSONLD fails to load. The test page here has
an encoding issue that causes the oembed jsonld to fail to load
and then readability fails to parse the date.
 2022-03-13 21:23:51 -07:00
Jackson Harper
78660c886d rm debug 2022-03-13 09:06:15 -07:00
Jackson Harper
a874482d11 Dont perform an extra query for isArchived 2022-03-13 09:00:25 -07:00
Jackson Harper
bea7d084c4 SetClaims when creating an email article 2022-03-09 19:45:52 -08:00
Jackson Harper
f7814a0c4a Remove unused function 2022-03-09 19:45:31 -08:00
Jackson Harper
b0fe9059a9 Dont try to generate highlight URL previews until share is re-enabled 2022-03-09 09:49:03 -08:00
Jackson Harper
a5a42a36e7
Merge pull request #206 from omnivore-app/dependabot/npm_and_yarn/opentelemetry/exporter-jaeger-1.0.1 
Bump @opentelemetry/exporter-jaeger from 0.24.0 to 1.0.1
 2022-03-08 21:00:53 -08:00
dependabot[bot]
9bda6ede06
Bump @opentelemetry/exporter-jaeger from 0.24.0 to 1.0.1 
Bumps [@opentelemetry/exporter-jaeger](https://github.com/open-telemetry/opentelemetry-js) from 0.24.0 to 1.0.1.
- [Release notes](https://github.com/open-telemetry/opentelemetry-js/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-js/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-js/compare/v0.24.0...stable/v1.0.1)

---
updated-dependencies:
- dependency-name: "@opentelemetry/exporter-jaeger"
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-03-09 04:45:18 +00:00
Jackson Harper
384a0771a7
Merge pull request #200 from omnivore-app/fix/validate-urls 
Use the validateUrl method to validate URLs
 2022-03-08 20:28:33 -08:00
Jackson Harper
c45c408c14 Fix formatting 2022-03-08 15:27:05 -08:00
Jackson Harper
e8fca4a7a9 Remove debug line 2022-03-08 15:20:24 -08:00
Jackson Harper
354fe36dd5 Add a test for save email 2022-03-08 15:17:11 -08:00
Jackson Harper
26dadab4aa rm debug, we dont need to set claims on create 2022-03-08 15:10:01 -08:00
Jackson Harper
b982bf34d6 SetClaims on userArticle create/update in saveEmail 2022-03-08 14:43:10 -08:00
Jackson Harper
9ae81d7394 Add extra debugging on newsletter save errors 2022-03-08 14:00:35 -08:00
Jackson Harper
2184c2a8d3 Parse online URLs for beehiiv newsletters 2022-03-07 15:49:44 -08:00
Jackson Harper
2cb5cc065a Add support for identifying newsletters hosted on beehiiv.com 2022-03-07 15:23:56 -08:00
Jackson Harper
bba094fbcd Add tests for beehiiv.com hosted newsletters 2022-03-07 15:05:38 -08:00
Jackson Harper
be7e36ffed Use the validateUrl method to validate URLs 2022-03-07 14:34:32 -08:00
Jackson Harper
ae329d2651 Add test data file 2022-03-03 19:47:37 -08:00
Jackson Harper
b6fd3e786e Fix parsing authors from page metadata 2022-03-03 19:40:02 -08:00
Jackson Harper
49092b707d Remove async 2022-03-03 19:31:51 -08:00
Jackson Harper
05373ba3c7 add methid to parse content metadata 2022-03-03 17:10:06 -08:00
Jackson Harper
c2e08d0e8f Fetch title and author from page metadate if possible 2022-03-03 15:20:58 -08:00
Jackson Harper
b17f22949c Use the readability title if available 2022-03-03 14:21:30 -08:00
Jackson Harper
a4bb67deee Use the readability byline for email authors if available 2022-03-03 14:21:03 -08:00
Jackson Harper
b326a5f8e7 Add more matches on substack icons 2022-03-03 13:42:30 -08:00
Jackson Harper
484cd78ac5 prettier 2022-03-02 23:14:10 -08:00
Jackson Harper
65ce8353dc Attempt to pull URLs for probable newsletter emails out of content 2022-03-02 23:09:10 -08:00
Jackson Harper
21329949e5 Fix to email address in saveNewsletterEmail 2022-03-02 22:06:15 -08:00
Jackson Harper
02d505a7e2 Add some debugging 2022-03-02 21:39:10 -08:00
Jackson Harper
8fe9827ad2 If an email appears to be a newsletter at it to the library 2022-03-02 20:47:53 -08:00
Jackson Harper
65cc666579 Pass HTML instead of a JSDOM into isProbablyNewsletter to better encapsulate 2022-03-02 20:38:11 -08:00
Jackson Harper
c4e237927d Allow any on GCP func 2022-03-02 20:27:40 -08:00
Jackson Harper
e660e41ed5 Fix warnings 2022-03-02 19:49:10 -08:00
Jackson Harper
9206230659 Better name for the save newsletter service 2022-03-02 19:45:28 -08:00
Jackson Harper
d66c114a7d Remove file, this is moved into the parser 2022-03-02 16:36:36 -08:00
Jackson Harper
b5f9478350 Dont mutate function input 2022-03-02 16:34:56 -08:00
Jackson Harper
f7f83fe080 New function to determine if an HTML blob is probably a newsletter based on its content 2022-03-02 16:31:15 -08:00
dependabot[bot]
29c941f4ae
Bump apollo-datasource from 0.7.3 to 3.3.1 
Bumps [apollo-datasource](https://github.com/apollographql/apollo-server/tree/HEAD/packages/apollo-datasource) from 0.7.3 to 3.3.1.
- [Release notes](https://github.com/apollographql/apollo-server/releases)
- [Changelog](https://github.com/apollographql/apollo-server/blob/main/CHANGELOG.md)
- [Commits](https://github.com/apollographql/apollo-server/commits/apollo-datasource@3.3.1/packages/apollo-datasource)

---
updated-dependencies:
- dependency-name: apollo-datasource
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-03-01 23:19:04 +00:00
Jackson Harper
505f888e37 Update to latest intercom client 2022-03-01 11:20:31 -08:00
dependabot[bot]
cadccfc7e1
Bump intercom-client from 2.11.2 to 3.1.4 
Bumps [intercom-client](https://github.com/intercom/intercom-node) from 2.11.2 to 3.1.4.
- [Release notes](https://github.com/intercom/intercom-node/releases)
- [Commits](https://github.com/intercom/intercom-node/commits/v3.1.4)

---
updated-dependencies:
- dependency-name: intercom-client
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-03-01 18:40:33 +00:00
Hongbo Wu
fc9aa9452c
Add a flag in readability to retain table elements in newsletter emails (#152) 
* add a flag in readability to retain table elements in newsletter emails

* remove header of axios newsletters
 2022-03-01 11:49:38 +08:00
dependabot[bot]
b2faf8a78d
Bump @opentelemetry/instrumentation-express from 0.24.0 to 0.28.0 
Bumps [@opentelemetry/instrumentation-express](https://github.com/open-telemetry/opentelemetry-js-contrib) from 0.24.0 to 0.28.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-js-contrib/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-js-contrib/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-js-contrib/compare/v0.24.0...contrib-test-utils-v0.28.0)

---
updated-dependencies:
- dependency-name: "@opentelemetry/instrumentation-express"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-02-28 21:27:15 +00:00
Hongbo Wu
7bf454ae91
use dataloader to fetch all labels of a list of linkIds in a single q… (#133) 
* use dataloader to fetch all labels of a list of linkIds in a single query and cached

* add labels in GQL query in frontend
 2022-02-28 12:13:26 +08:00
Jackson Harper
328ebc48cb Apply code block highlighting before running DOM clean 2022-02-26 19:04:32 -08:00
Jackson Harper
e0b8cfc938 Import highlightjs types 
We still need these dev packages because of the way our docker
builds are setup.
 2022-02-26 17:37:01 -08:00
Jackson Harper
fd39923907 Prettier improvements 2022-02-26 16:38:43 -08:00
Jackson Harper
84fbc9cd27 Add code highlighting using highlight.js 2022-02-26 14:57:59 -08:00
Satindar Dhillon
e27c2bdd8b generate swift gql code 2022-02-25 10:09:17 -08:00
Hongbo Wu
b9d581166e fix merge conflicts 2022-02-24 14:56:00 +08:00
Hongbo Wu
42f2cffdf8 remove debug log 2022-02-24 14:47:33 +08:00
Hongbo Wu
276205d52a add linkId and labels in article type 2022-02-24 14:44:49 +08:00
Hongbo Wu
c7841b8e8a remove duplicate links because of join with link_labels table 2022-02-24 14:44:49 +08:00
Hongbo Wu
fc18004a5d add labels in search query 2022-02-24 14:44:49 +08:00
Hongbo Wu
edb9339897 add labels and link_labels table name 2022-02-24 14:44:49 +08:00
Hongbo Wu
5651c92efe ingore case when checking name uniqueness 2022-02-24 14:44:49 +08:00
Hongbo Wu
9599c98c77 add label filters in search query 2022-02-24 14:44:49 +08:00
Hongbo Wu
fc37d06fd2 update params variable 2022-02-24 14:44:49 +08:00
Hongbo Wu
d33b213092 add setLabels api and tests 2022-02-24 14:44:48 +08:00
Hongbo Wu
9150e7dbc2 add tests for create/delete labels 2022-02-24 14:44:48 +08:00
Hongbo Wu
e74f334db9 update entity 2022-02-24 14:44:48 +08:00
Jackson Harper
6e6745a2b7 Use version 6.1.0 for normalize-url which switches to modules in v7 2022-02-23 10:28:28 -08:00
dependabot[bot]
385cfdad71
Bump normalize-url from 5.3.1 to 7.0.3 
Bumps [normalize-url](https://github.com/sindresorhus/normalize-url) from 5.3.1 to 7.0.3.
- [Release notes](https://github.com/sindresorhus/normalize-url/releases)
- [Commits](https://github.com/sindresorhus/normalize-url/commits/v7.0.3)

---
updated-dependencies:
- dependency-name: normalize-url
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-02-23 18:14:46 +00:00
Jackson Harper
7c9c75836b Update the monitor packages as these are peer dependencies 2022-02-23 09:26:48 -08:00
Jackson Harper
5e2bba3231 Upgrade opentelemetry packages 2022-02-22 12:42:19 -08:00
dependabot[bot]
4a9d24025b Bump @opentelemetry/node from 0.22.0 to 0.24.0 
Bumps [@opentelemetry/node](https://github.com/open-telemetry/opentelemetry-js) from 0.22.0 to 0.24.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-js/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-js/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-js/compare/v0.22.0...v0.24.0)

---
updated-dependencies:
- dependency-name: "@opentelemetry/node"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-02-22 12:42:19 -08:00
Jackson Harper
5f6f3ba3b2 Fix warnings in the API package 2022-02-21 16:46:10 -08:00
Jackson Harper
a276af5264 Remove extra white space 2022-02-21 13:55:09 -08:00
Jackson Harper
cf6733c70f When fetching the library item list dont add content/originalHtml from DB 2022-02-21 13:55:09 -08:00
Jackson Harper
5af4899415 Use null instead of undefined when unarchiving links 
This fixes unarchiving content by re-saving it via the Save*
API methods.

Setting to undefined is not the same as null in this case, and
will result in archived_at being non-null and the link then
being excluded from the GetArticles query.
 2022-02-21 11:26:22 -08:00
Hongbo Wu
4c84c40f97
add slug to the link (#84) 2022-02-18 10:43:34 +08:00
Jackson Harper
05aa5d737e
Merge pull request #82 from omnivore-app/feature/bloomberg-newsletters 
Add support for Bloomberg newsletter
 2022-02-18 01:08:46 +08:00
Hongbo Wu
9431e80be8 add parser 2022-02-17 20:17:02 +08:00
Hongbo Wu
e3839647e1 parse bloomberg newsletter 2022-02-17 20:14:06 +08:00
Hongbo Wu
ad8bca3d84 use subject as title 2022-02-17 18:59:07 +08:00
Hongbo Wu
7229c64da0
fix: Should not get fake uploading url in demo (#79) 
* fix not getting fake uploading url in demo

* use internal service endpoint env

* update internal svc endpoint
 2022-02-17 13:34:50 +08:00
Hongbo Wu
bd8178a464 add support for bloomberg newsletters 2022-02-17 13:19:58 +08:00
Jackson Harper
3a27681534
Merge pull request #74 from omnivore-app/chore/upgrade-gql-typegen-2.6.2 
Upgrade gql-codegen, regenerate GQL schema
 2022-02-17 04:03:31 +08:00
Jackson Harper
e80eb7ab06 Upgrade gql-codegen, regenerate GQL schema 2022-02-16 11:32:13 -08:00
Hongbo Wu
c1ffd9abdf fix sanitize bug by having different SanitizedString type name when creating new instances 2022-02-16 09:33:33 -08:00
Jackson Harper
e327b7c57f Expect a 400 error for failed GQL directives 2022-02-16 09:33:33 -08:00
Jackson Harper
bf305bef39 Make sanitize test more explicit 2022-02-16 09:33:33 -08:00
Jackson Harper
c8abf87367 Cast the directive types 2022-02-16 09:33:33 -08:00
Jackson Harper
6d5554dd5f Implement the sanitize directive in applo 3.0 2022-02-16 09:33:33 -08:00
Jackson Harper
7e35281bd1 createApp is no longer async, linting 2022-02-16 09:33:33 -08:00
Jackson Harper
fb224d60c4 Remove subscriptions, comment out the sanitize directive 2022-02-16 09:33:30 -08:00
Jackson Harper
df2b36b33a Upgrade apollo-server-express, remove connections/subscriptions 2022-02-16 09:32:48 -08:00
Jackson Harper
5a457479e2
Merge pull request #64 from omnivore-app/feature/handle-pdf-attachment 
add pdf attachment to library
 2022-02-17 00:55:27 +08:00
Hongbo Wu
d53e94aeb2 add tests for create article 2022-02-16 16:14:14 +08:00
Hongbo Wu
77ba7531f3 add tests for upload 2022-02-16 15:26:11 +08:00
Hongbo Wu
9460197132 add more logs for debugging 2022-02-16 11:34:01 +08:00
Hongbo Wu
b826a101c5 add pdf attachment create-article 2022-02-16 09:24:16 +08:00
Jackson Harper
340d7903b3
Merge pull request #67 from omnivore-app/fix/upgrade-gql-typegen 
Upgrade gql-typegen and regen types
 2022-02-16 02:52:02 +08:00
Jackson Harper
c3308085ee Upgrade gql-typegen and regen types 2022-02-15 10:42:05 -08:00
Jackson Harper
06a38ae407
Merge pull request #55 from omnivore-app/dependabot/npm_and_yarn/opentelemetry/instrumentation-grpc-0.27.0 
Bump @opentelemetry/instrumentation-grpc from 0.22.0 to 0.27.0
 2022-02-16 02:05:58 +08:00
Hongbo Wu
5d95e6c734 add pdf attachment upload 2022-02-15 22:22:13 +08:00
Jackson Harper
49e436b10c Remove unneeded installs from API docker file 
These were needed when Chromium was installed on this image.
 2022-02-14 21:52:04 -08:00
dependabot[bot]
f93a3e3070
Bump @opentelemetry/instrumentation-grpc from 0.22.0 to 0.27.0 
Bumps [@opentelemetry/instrumentation-grpc](https://github.com/open-telemetry/opentelemetry-js) from 0.22.0 to 0.27.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-js/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-js/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-js/compare/v0.22.0...experimental/v0.27.0)

---
updated-dependencies:
- dependency-name: "@opentelemetry/instrumentation-grpc"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-02-15 05:22:49 +00:00
Hongbo Wu
e85273d87a
add support for axios newsletters (#49) 
* add support for axios newsletters

* fix raw url

* fix getting newsletter url from axios

* add test for url

* add axios parser
 2022-02-15 11:56:03 +08:00
Jackson Harper
65cd8a95eb Better handling of situations where we fail to set the cookie during auth 2022-02-14 16:04:23 -08:00
Jackson Harper
165085a639 Handle upgraded axios API for headers 2022-02-14 15:48:36 -08:00
Jackson Harper
238a2f0e97
Merge pull request #56 from omnivore-app/dependabot/npm_and_yarn/axios-0.26.0 
Bump axios from 0.21.4 to 0.26.0
 2022-02-15 07:25:37 +08:00
Jackson Harper
a23ae33264
Merge pull request #54 from omnivore-app/dependabot/npm_and_yarn/opentelemetry/instrumentation-dns-0.27.1 
Bump @opentelemetry/instrumentation-dns from 0.22.0 to 0.27.1
 2022-02-15 07:24:16 +08:00
Jackson Harper
952e110ccd Remove unneeded values from api env example file 2022-02-14 14:56:07 -08:00