prehistoric-systems/omnivore
1
0
Fork 0
mirror of https://github.com/omnivore-app/omnivore.git synced 2026-03-11 08:54:26 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
12057 commits 201 branches 31 tags 432 MiB
Commit graph

176 commits

Author SHA1 Message Date
Hongbo Wu
795ca47414 Save wordsCount when page is created 2022-09-06 10:19:58 +08:00
Hongbo Wu
3953a9098e Send SSML items in the synthesis API 2022-09-02 15:33:00 +08:00
Hongbo Wu
def8f28138 Add Azure Speech Service dependencies 2022-08-18 19:24:38 +08:00
Hongbo Wu
77570aa5ab Synthesize text to speech with azure API 2022-08-18 19:24:36 +08:00
Hongbo Wu
bca0f5ed59 Create Amazon Polly text to speech integration 2022-08-18 19:23:41 +08:00
dependabot[bot]
9ded821d85
Bump @sentry/integrations from 6.19.6 to 7.10.0 
Bumps [@sentry/integrations](https://github.com/getsentry/sentry-javascript) from 6.19.6 to 7.10.0.
- [Release notes](https://github.com/getsentry/sentry-javascript/releases)
- [Changelog](https://github.com/getsentry/sentry-javascript/blob/master/CHANGELOG.md)
- [Commits](https://github.com/getsentry/sentry-javascript/compare/6.19.6...7.10.0)

---
updated-dependencies:
- dependency-name: "@sentry/integrations"
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-08-15 02:16:56 +00:00
Jackson Harper
5de2d65ce4
Merge pull request #1052 from omnivore-app/dependabot/npm_and_yarn/sentry/tracing-7.9.0 
Bump @sentry/tracing from 5.30.0 to 7.9.0
 2022-08-15 10:14:54 +08:00
dependabot[bot]
858c91d814
Bump @sentry/tracing from 5.30.0 to 7.9.0 
Bumps [@sentry/tracing](https://github.com/getsentry/sentry-javascript) from 5.30.0 to 7.9.0.
- [Release notes](https://github.com/getsentry/sentry-javascript/releases)
- [Changelog](https://github.com/getsentry/sentry-javascript/blob/master/CHANGELOG.md)
- [Commits](https://github.com/getsentry/sentry-javascript/compare/5.30.0...7.9.0)

---
updated-dependencies:
- dependency-name: "@sentry/tracing"
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-08-08 21:51:50 +00:00
Hongbo Wu
73bf695cf3 Add function to get highlight start location 2022-08-05 16:08:20 +08:00
Hongbo Wu
469910f2bd Add addressparser types dependency 2022-07-28 11:37:14 +08:00
Hongbo Wu
ec9247bc8a Add addressparser dependency 2022-07-28 10:44:44 +08:00
Jackson Harper
ca33f31ebb
Merge pull request #1014 from omnivore-app/dependabot/npm_and_yarn/opentelemetry/instrumentation-pg-0.30.0 
Bump @opentelemetry/instrumentation-pg from 0.29.0 to 0.30.0
 2022-07-26 16:11:46 -07:00
Hongbo Wu
7339a0daf3 add sinon and sinon-chai for faking sendMail 2022-07-25 15:12:55 -07:00
dependabot[bot]
9dbf9bce67
Bump @opentelemetry/instrumentation-pg from 0.29.0 to 0.30.0 
Bumps [@opentelemetry/instrumentation-pg](https://github.com/open-telemetry/opentelemetry-js-contrib) from 0.29.0 to 0.30.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-js-contrib/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-js-contrib/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-js-contrib/compare/host-metrics-v0.29.0...contrib-test-utils-v0.30.0)

---
updated-dependencies:
- dependency-name: "@opentelemetry/instrumentation-pg"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-07-25 21:29:38 +00:00
dependabot[bot]
b86d9a08b9
Bump @google-cloud/logging-winston from 4.2.2 to 5.1.1 
Bumps [@google-cloud/logging-winston](https://github.com/googleapis/nodejs-logging-winston) from 4.2.2 to 5.1.1.
- [Release notes](https://github.com/googleapis/nodejs-logging-winston/releases)
- [Changelog](https://github.com/googleapis/nodejs-logging-winston/blob/main/CHANGELOG.md)
- [Commits](https://github.com/googleapis/nodejs-logging-winston/compare/v4.2.2...v5.1.1)

---
updated-dependencies:
- dependency-name: "@google-cloud/logging-winston"
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-07-18 21:59:44 +00:00
Jackson Harper
b33f87c982
Merge pull request #943 from omnivore-app/dependabot/npm_and_yarn/opentelemetry/instrumentation-graphql-0.29.0 
Bump @opentelemetry/instrumentation-graphql from 0.24.0 to 0.29.0
 2022-07-12 09:32:35 -07:00
Jackson Harper
f7a023939d
Merge pull request #940 from omnivore-app/feature/allow-fetching-original 2022-07-11 22:08:27 -07:00
dependabot[bot]
87e4ea0d5c
Bump @opentelemetry/instrumentation-graphql from 0.24.0 to 0.29.0 
Bumps [@opentelemetry/instrumentation-graphql](https://github.com/open-telemetry/opentelemetry-js-contrib) from 0.24.0 to 0.29.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-js-contrib/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-js-contrib/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-js-contrib/compare/v0.24.0...host-metrics-v0.29.0)

---
updated-dependencies:
- dependency-name: "@opentelemetry/instrumentation-graphql"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-07-11 21:33:57 +00:00
dependabot[bot]
73f9f64aea
Bump @google-cloud/monitoring from 2.3.5 to 3.0.0 
Bumps [@google-cloud/monitoring](https://github.com/googleapis/nodejs-monitoring) from 2.3.5 to 3.0.0.
- [Release notes](https://github.com/googleapis/nodejs-monitoring/releases)
- [Changelog](https://github.com/googleapis/nodejs-monitoring/blob/main/CHANGELOG.md)
- [Commits](https://github.com/googleapis/nodejs-monitoring/compare/v2.3.5...v3.0.0)

---
updated-dependencies:
- dependency-name: "@google-cloud/monitoring"
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-07-11 02:50:23 +00:00
Jackson Harper
9f007d72d2
Merge pull request #892 from omnivore-app/dependabot/npm_and_yarn/opentelemetry/instrumentation-dns-0.29.0 
Bump @opentelemetry/instrumentation-dns from 0.27.1 to 0.29.0
 2022-07-10 19:48:26 -07:00
Jackson Harper
9feeb80cf3 Allow fetching original content 
This is useful when debugging
 2022-07-09 14:35:14 -07:00
Jackson Harper
c94d5db259
Merge pull request #889 from omnivore-app/dependabot/npm_and_yarn/axios-0.27.2 
Bump axios from 0.26.0 to 0.27.2
 2022-07-08 13:48:27 -07:00
dependabot[bot]
a4219ac15c
Bump googleapis from 100.0.0 to 105.0.0 
Bumps [googleapis](https://github.com/googleapis/google-api-nodejs-client) from 100.0.0 to 105.0.0.
- [Release notes](https://github.com/googleapis/google-api-nodejs-client/releases)
- [Changelog](https://github.com/googleapis/google-api-nodejs-client/blob/main/CHANGELOG.md)
- [Commits](https://github.com/googleapis/google-api-nodejs-client/compare/googleapis-v100.0.0...googleapis-v105.0.0)

---
updated-dependencies:
- dependency-name: googleapis
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-07-04 21:40:35 +00:00
dependabot[bot]
33d94edfac
Bump @opentelemetry/instrumentation-dns from 0.27.1 to 0.29.0 
Bumps [@opentelemetry/instrumentation-dns](https://github.com/open-telemetry/opentelemetry-js-contrib) from 0.27.1 to 0.29.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-js-contrib/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-js-contrib/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-js-contrib/compare/host-metrics-v0.27.1...host-metrics-v0.29.0)

---
updated-dependencies:
- dependency-name: "@opentelemetry/instrumentation-dns"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-06-27 21:34:28 +00:00
dependabot[bot]
e91f25e58c
Bump axios from 0.26.0 to 0.27.2 
Bumps [axios](https://github.com/axios/axios) from 0.26.0 to 0.27.2.
- [Release notes](https://github.com/axios/axios/releases)
- [Changelog](https://github.com/axios/axios/blob/v0.27.2/CHANGELOG.md)
- [Commits](https://github.com/axios/axios/compare/v0.26.0...v0.27.2)

---
updated-dependencies:
- dependency-name: axios
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-06-27 21:31:14 +00:00
Jackson Harper
189f95d14b Use mocha-unfunk-reporter with the API tests 
This should give us better reporting of failures in GitHub
Actions.
 2022-06-24 11:55:29 -07:00
dependabot[bot]
e22b575759
Bump @opentelemetry/core from 0.24.0 to 1.3.1 
Bumps [@opentelemetry/core](https://github.com/open-telemetry/opentelemetry-js) from 0.24.0 to 1.3.1.
- [Release notes](https://github.com/open-telemetry/opentelemetry-js/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-js/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-js/commits)

---
updated-dependencies:
- dependency-name: "@opentelemetry/core"
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-06-13 21:26:08 +00:00
dependabot[bot]
cfbfed300f
Bump @opentelemetry/instrumentation-grpc from 0.27.0 to 0.29.2 
Bumps [@opentelemetry/instrumentation-grpc](https://github.com/open-telemetry/opentelemetry-js) from 0.27.0 to 0.29.2.
- [Release notes](https://github.com/open-telemetry/opentelemetry-js/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-js/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-js/commits)

---
updated-dependencies:
- dependency-name: "@opentelemetry/instrumentation-grpc"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-06-09 16:34:22 +00:00
dependabot[bot]
726e918791
Bump @opentelemetry/instrumentation-pg from 0.24.0 to 0.29.0 
Bumps [@opentelemetry/instrumentation-pg](https://github.com/open-telemetry/opentelemetry-js-contrib) from 0.24.0 to 0.29.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-js-contrib/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-js-contrib/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-js-contrib/compare/v0.24.0...contrib-test-utils-v0.29.0)

---
updated-dependencies:
- dependency-name: "@opentelemetry/instrumentation-pg"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-06-06 21:21:03 +00:00
Hongbo Wu
304fe70113 Fix tests 2022-06-01 10:30:23 +08:00
dependabot[bot]
cf516bba54
Bump ts-loader from 8.3.0 to 9.3.0 
Bumps [ts-loader](https://github.com/TypeStrong/ts-loader) from 8.3.0 to 9.3.0.
- [Release notes](https://github.com/TypeStrong/ts-loader/releases)
- [Changelog](https://github.com/TypeStrong/ts-loader/blob/main/CHANGELOG.md)
- [Commits](https://github.com/TypeStrong/ts-loader/compare/v8.3.0...v9.3.0)

---
updated-dependencies:
- dependency-name: ts-loader
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-05-23 21:20:01 +00:00
Hongbo Wu
0b11c31317 Add linkedom dependency in packages/api 2022-05-10 18:31:25 +08:00
Hongbo Wu
93ed1936f0 Remove jsdom in packages/api 2022-05-10 17:12:52 +08:00
dependabot[bot]
1b27bf1f4b
Bump cookie from 0.4.2 to 0.5.0 
Bumps [cookie](https://github.com/jshttp/cookie) from 0.4.2 to 0.5.0.
- [Release notes](https://github.com/jshttp/cookie/releases)
- [Changelog](https://github.com/jshttp/cookie/blob/master/HISTORY.md)
- [Commits](https://github.com/jshttp/cookie/compare/v0.4.2...v0.5.0)

---
updated-dependencies:
- dependency-name: cookie
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-04-21 01:43:26 +00:00
Hongbo Wu
83cdec42f7
Add rate limiter for api request (#454) 
* Add rate limiter for api request

* apply rate limit on Demo/Prod
 2022-04-21 09:41:33 +08:00
Jackson Harper
035b61665d
Merge pull request #376 from omnivore-app/dependabot/npm_and_yarn/googleapis-100.0.0 
Bump googleapis from 59.0.0 to 100.0.0
 2022-04-12 08:47:11 -07:00
Hongbo Wu
ae0d1dd2ee
Feature/search highlights backend (#395) 
* add highlight mappings

* return highlight in resolvers

* temporarily skip highlight tests

* add test for getting highlights

* update merge highlight

* separate elastic methods

* roll back merge highlight test

* add highlight to elastic script

* update delete highlight in elastic

* migrate highlight data from postgres to elastic

* rescue not found exception when page is not found in the migration script

* exclude highlights in searching pages results

* search pages with highlights only with has:highlight query

* add search endpoint to search pages or highlights

* reduce code smell in search api

* fix rebase error

* fix tests

* add test for search highlight

* add test for new search endpoint

* add labels to search results

* update schema

* update search query

* fix update/share highlights

* fix rebase error

* fix tests

* add highlight model in elastic

* add savedAt and publishedAt date range in search query

* add sort by updated and recently read

* fix tests

* close db connection when tests are done

* test github action

* revert github action test

* fix rebase error

* add docker-compose for api-test

* remove unused env

* remove highlights with no page attached to

* allow get_articles resolver to search for query so we can merge it without web changes
 2022-04-12 12:31:08 +08:00
Jackson Harper
26bff1b25d Pin yarn version w/volta 2022-04-06 10:06:24 -07:00
Jackson Harper
63a374b31d package syntax 2022-04-06 10:05:06 -07:00
Jackson Harper
1bac06107d Update volta version 2022-04-06 10:04:04 -07:00
Hongbo Wu
7569e988bf
upgrade typeorm to 3.0 (#359) 
* upgrade typeorm to 3.0

* use new datasource object in typeorm 3

* fix tests

* fix tests

* migrate before creating connection

* fail the test if migration failed
 2022-04-06 10:32:41 +08:00
dependabot[bot]
8df6e750dc
Bump googleapis from 59.0.0 to 100.0.0 
Bumps [googleapis](https://github.com/googleapis/google-api-nodejs-client) from 59.0.0 to 100.0.0.
- [Release notes](https://github.com/googleapis/google-api-nodejs-client/releases)
- [Changelog](https://github.com/googleapis/google-api-nodejs-client/blob/main/CHANGELOG.md)
- [Commits](https://github.com/googleapis/google-api-nodejs-client/compare/v59.0.0...googleapis-v100.0.0)

---
updated-dependencies:
- dependency-name: googleapis
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-04-04 22:36:25 +00:00
Jackson Harper
62c3bcd968
Merge pull request #338 from omnivore-app/dependabot/npm_and_yarn/opentelemetry/semantic-conventions-1.0.1 
Bump @opentelemetry/semantic-conventions from 0.24.0 to 1.0.1
 2022-04-04 15:35:51 -07:00
dependabot[bot]
3b4254a711
Bump luxon from 1.28.0 to 2.3.1 
Bumps [luxon](https://github.com/moment/luxon) from 1.28.0 to 2.3.1.
- [Release notes](https://github.com/moment/luxon/releases)
- [Changelog](https://github.com/moment/luxon/blob/master/CHANGELOG.md)
- [Commits](https://github.com/moment/luxon/compare/1.28.0...2.3.1)

---
updated-dependencies:
- dependency-name: luxon
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-04-04 17:55:59 +00:00
Jackson Harper
46c5b8c82a Another attempt at slimming docker file 2022-04-04 10:42:04 -07:00
Jackson Harper
946734d76c Revert docker changes 
These were creating fatter images than we want because of the
lerna dependencies.
 2022-04-04 09:59:18 -07:00
Jackson Harper
b8b471d369 Move @types into dev 2022-04-04 08:17:00 -07:00
Jackson Harper
fc17daa546 Remove unneeded mocha params 2022-04-02 17:02:11 -07:00
Jackson Harper
da28998130 Pull ul list out of newsletter blurb 
next/react doesnt want child elements of the paragraphs

Improve formatting

Improve wording

Use buttons in the subscribe directly blocks

Simplify docs on setting up forwarding rules

Add extra padding on bottom of help docs

Remove unused style

Add emails help page

Improve formatting

Prefetch page content on iOS

Reduce the reader overly length now that items are precached

Add invalidation when highlights are added to items

fix missing index_settings.json file in api dockerfile for creating elastic index (#363)

Handle full email address objects in the to param from sendgrid

These come in a format like:

"jacksonh-dfdf@inbox.omnivore.app" <jacksonh-dfdf@inbox-demo.omnivore.app>

New IDs for short highlights because they dont cascade delete now

Testing CI issues

Simplify test

CI test

Use promises for async tests

Temporarily remove test to debug CI

Re-enable

re-enable test, return error

Specify a userId when looking up saved email pages

create a unique url for newsletters without a URL

Use 500ms on page test timeouts

Increase timeout

Dont use deep equal to match newsletter label

Run just the labels API

Run against just the newsletter emails

Run without the page tests

Fix

Set the allow uncaught flag

Remove highlight tests

Remove newsletters tests

more resolver tests

Remove newsetter tests

Comment out resolver tests

Use nock for external requests in tests

Specify puppeteer url for tests

Comment out more tests

uncomment tests

re-enable

re-enable email test

Re-disable

Re-enable one pdf attachment test

Re-disable pdf attachment test

Use promises on setTimeout tests

rm label tests

mv label tests into a context

Comment out pdf tests

Comment out pdf tests

Async test

Async wrappers

Delay when creating test pages

More debugging

Unique short ids

Remove potentially problematic test

Fetch page before returning for test

handler in before block

more debugging

More debugging

Move errors checks into contexts

Use a context when saving newsletters to force index refresh

Prettier fix

Fix newsletter label check, remove setTimeout

Re-enable test

timeout on pdf router handler

Fix method call

comment out PDF test

Unique fake username

Comment out PDF test

Debugging signed urls

Re-enable

New email

pdf test

PDF tests

Comment out pdf test

Add nock stubs for email URLs

Use full address for PDF test

Remove debug

Use full email addresses
 2022-04-02 16:56:24 -07:00
dependabot[bot]
d0af8b2186
Bump @opentelemetry/semantic-conventions from 0.24.0 to 1.0.1 
Bumps [@opentelemetry/semantic-conventions](https://github.com/open-telemetry/opentelemetry-js) from 0.24.0 to 1.0.1.
- [Release notes](https://github.com/open-telemetry/opentelemetry-js/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-js/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-js/compare/v0.24.0...stable/v1.0.1)

---
updated-dependencies:
- dependency-name: "@opentelemetry/semantic-conventions"
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-03-28 21:28:57 +00:00
Jackson Harper
5036924b21
Merge pull request #280 from omnivore-app/dependabot/npm_and_yarn/jsdom-19.0.0 
Bump jsdom from 16.7.0 to 19.0.0
 2022-03-25 09:08:55 -07:00
dependabot[bot]
3ec6f22199
Bump jsdom from 16.7.0 to 19.0.0 
Bumps [jsdom](https://github.com/jsdom/jsdom) from 16.7.0 to 19.0.0.
- [Release notes](https://github.com/jsdom/jsdom/releases)
- [Changelog](https://github.com/jsdom/jsdom/blob/master/Changelog.md)
- [Commits](https://github.com/jsdom/jsdom/compare/16.7.0...19.0.0)

---
updated-dependencies:
- dependency-name: jsdom
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-03-23 18:51:08 +00:00
dependabot[bot]
783118a2e4
Bump @sentry/integrations from 5.30.0 to 6.19.1 
Bumps [@sentry/integrations](https://github.com/getsentry/sentry-javascript) from 5.30.0 to 6.19.1.
- [Release notes](https://github.com/getsentry/sentry-javascript/releases)
- [Changelog](https://github.com/getsentry/sentry-javascript/blob/master/CHANGELOG.md)
- [Commits](https://github.com/getsentry/sentry-javascript/compare/5.30.0...6.19.1)

---
updated-dependencies:
- dependency-name: "@sentry/integrations"
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-03-21 21:19:05 +00:00
Jackson Harper
ff1200f3a1 Use html decoding when getting values from fetched oembed 
If we fetch oembed data from an external source, instead of
handling it in readabilityjs we need to html decode it.
 2022-03-16 15:29:42 -07:00
Jackson Harper
88103cba9f
Merge pull request #162 from omnivore-app/dependabot/npm_and_yarn/apollo-datasource-3.3.1 
Bump apollo-datasource from 0.7.3 to 3.3.1
 2022-03-16 11:03:26 -07:00
Jackson Harper
e652a6ea8c
Rebased version of the elastic PR (#225) 
* Add elastic to our docker compose

* add AND/OR/NOT search operations

* add elastic and create article in elastic

* change error code when elastic throws error

* add search pages in elastic

* add search by labels

* Add elastic to GitHub Action

* Update elastic version

* Fix port for elastic

* add url in search query

* Set elastic features when running tests

* add debug logs

* Use localhost instead of service hostname

* refresh elastic after create/update

* update search labels query

* add typescript support

* search pages in elastic

* fix search queries

* use elastic for saving page

* fix test failure

* update getArticle api to use elastic

* use generic get page function

* add elastic migration python script

* fix bulk helper param

* save elastic page id in article_saving_request instead of postgres article_id

* fix page archiving and deleting

* add tests for deleteArticle

* remove custom date type in elastic mappings which not exist in older version of elastic

* fix timestamp format issue

* add tests for save reading progress

* add tests for save file

* optimize search results

* add alias to index

* update migration script to receive env var as params

* Add failing test to validate we don't decrease reading progress

This test is failing with Elastic because we aren't fetching
the reading progress from elastic here, and are fetching it
from postgres.

* Rename readingProgress to readingProgressPercent

This is the name stored in elastic, so fixes issues pulling the
value out.

* Linting

* Add failing test for creating highlights w/elastic

This test fails because the highlight can't be looked up. Is there
a different ID we should be passing in to query for highlights,
or do we need to update the query to look for elastic_id?

* add tests code coverage threshold

* update nyc config

* include more files in test coverage

* change alias name

* update updateContent to update pages in elastic

* remove debug log

* fix createhighlight test

* search pages by alias in elastic

* update set labels and delete labels in elastic

* migration script enumeration

* make BULK_SIZE an env var

* fix pdf search indexing

* debug github action exit issue

* call pubsub when create/update/delete page in elastic

* fix json parsing bug and reduce reading data from file

* replace a depreciated pubsub api call

* debug github action exit issue

* debug github action exit issue

* add handler to upload elastic page data to GCS

* fix tests

* Use http_auth instead of basic_auth

* add index creation and existing postgres tables update in migration script

* fix a typo to connect to elastic

* rename readingProgress to readingProgressPercent

* migrate elastic_page_id in highlights and article_saving_request tables

* update migration script to include number of updated rows

* update db migration query

* read index mappings from file

* fix upload pages to gcs

* fix tests failure due to pageContext

* fix upload file id not exist error

* Handle savedAt & isArchived attributes w/out quering elastic

* Fix prettier issues

* fix content-type mismatching

* revert pageId to linkId because frontend was not deployed yet

* fix newsletters and attachment not saved in elastic

* put linkId in article for setting labels

* exclude orginalHtml in the result of searching to improve performace

* exclude content in the result of searching to improve performace

* remove score sorting

* do not refresh immediately to reduce searching and indexing time

* do not replace the backup data in gcs

* fix no article id defined in articleSavingRequest

* add logging of elastic api running time

* reduce home feed pagination size to 15

* reduce home feed pagination size to 10

* stop revalidating first page

* do not use a separate api to fetch reading progress

* Remove unused comment

* get reading progress if not exists

* replace ngram tokenizer with standard tokenizer

* fix tests

* remove .env.local

* add sort keyword in searching to sort by score

Co-authored-by: Hongbo Wu <hongbo@omnivore.app>
 2022-03-16 12:08:59 +08:00
dependabot[bot]
9bda6ede06
Bump @opentelemetry/exporter-jaeger from 0.24.0 to 1.0.1 
Bumps [@opentelemetry/exporter-jaeger](https://github.com/open-telemetry/opentelemetry-js) from 0.24.0 to 1.0.1.
- [Release notes](https://github.com/open-telemetry/opentelemetry-js/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-js/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-js/compare/v0.24.0...stable/v1.0.1)

---
updated-dependencies:
- dependency-name: "@opentelemetry/exporter-jaeger"
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-03-09 04:45:18 +00:00
dependabot[bot]
29c941f4ae
Bump apollo-datasource from 0.7.3 to 3.3.1 
Bumps [apollo-datasource](https://github.com/apollographql/apollo-server/tree/HEAD/packages/apollo-datasource) from 0.7.3 to 3.3.1.
- [Release notes](https://github.com/apollographql/apollo-server/releases)
- [Changelog](https://github.com/apollographql/apollo-server/blob/main/CHANGELOG.md)
- [Commits](https://github.com/apollographql/apollo-server/commits/apollo-datasource@3.3.1/packages/apollo-datasource)

---
updated-dependencies:
- dependency-name: apollo-datasource
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-03-01 23:19:04 +00:00
dependabot[bot]
cadccfc7e1
Bump intercom-client from 2.11.2 to 3.1.4 
Bumps [intercom-client](https://github.com/intercom/intercom-node) from 2.11.2 to 3.1.4.
- [Release notes](https://github.com/intercom/intercom-node/releases)
- [Commits](https://github.com/intercom/intercom-node/commits/v3.1.4)

---
updated-dependencies:
- dependency-name: intercom-client
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-03-01 18:40:33 +00:00
dependabot[bot]
b2faf8a78d
Bump @opentelemetry/instrumentation-express from 0.24.0 to 0.28.0 
Bumps [@opentelemetry/instrumentation-express](https://github.com/open-telemetry/opentelemetry-js-contrib) from 0.24.0 to 0.28.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-js-contrib/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-js-contrib/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-js-contrib/compare/v0.24.0...contrib-test-utils-v0.28.0)

---
updated-dependencies:
- dependency-name: "@opentelemetry/instrumentation-express"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-02-28 21:27:15 +00:00
Jackson Harper
e0b8cfc938 Import highlightjs types 
We still need these dev packages because of the way our docker
builds are setup.
 2022-02-26 17:37:01 -08:00
Jackson Harper
84fbc9cd27 Add code highlighting using highlight.js 2022-02-26 14:57:59 -08:00
Jackson Harper
6e6745a2b7 Use version 6.1.0 for normalize-url which switches to modules in v7 2022-02-23 10:28:28 -08:00
dependabot[bot]
385cfdad71
Bump normalize-url from 5.3.1 to 7.0.3 
Bumps [normalize-url](https://github.com/sindresorhus/normalize-url) from 5.3.1 to 7.0.3.
- [Release notes](https://github.com/sindresorhus/normalize-url/releases)
- [Commits](https://github.com/sindresorhus/normalize-url/commits/v7.0.3)

---
updated-dependencies:
- dependency-name: normalize-url
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-02-23 18:14:46 +00:00
Jackson Harper
7c9c75836b Update the monitor packages as these are peer dependencies 2022-02-23 09:26:48 -08:00
Jackson Harper
5e2bba3231 Upgrade opentelemetry packages 2022-02-22 12:42:19 -08:00
dependabot[bot]
4a9d24025b Bump @opentelemetry/node from 0.22.0 to 0.24.0 
Bumps [@opentelemetry/node](https://github.com/open-telemetry/opentelemetry-js) from 0.22.0 to 0.24.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-js/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-js/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-js/compare/v0.22.0...v0.24.0)

---
updated-dependencies:
- dependency-name: "@opentelemetry/node"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-02-22 12:42:19 -08:00
Jackson Harper
df2b36b33a Upgrade apollo-server-express, remove connections/subscriptions 2022-02-16 09:32:48 -08:00
dependabot[bot]
f93a3e3070
Bump @opentelemetry/instrumentation-grpc from 0.22.0 to 0.27.0 
Bumps [@opentelemetry/instrumentation-grpc](https://github.com/open-telemetry/opentelemetry-js) from 0.22.0 to 0.27.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-js/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-js/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-js/compare/v0.22.0...experimental/v0.27.0)

---
updated-dependencies:
- dependency-name: "@opentelemetry/instrumentation-grpc"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-02-15 05:22:49 +00:00
Jackson Harper
238a2f0e97
Merge pull request #56 from omnivore-app/dependabot/npm_and_yarn/axios-0.26.0 
Bump axios from 0.21.4 to 0.26.0
 2022-02-15 07:25:37 +08:00
dependabot[bot]
0f843809e3
Bump axios from 0.21.4 to 0.26.0 
Bumps [axios](https://github.com/axios/axios) from 0.21.4 to 0.26.0.
- [Release notes](https://github.com/axios/axios/releases)
- [Changelog](https://github.com/axios/axios/blob/master/CHANGELOG.md)
- [Commits](https://github.com/axios/axios/compare/v0.21.4...v0.26.0)

---
updated-dependencies:
- dependency-name: axios
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-02-14 21:35:58 +00:00
dependabot[bot]
77598cd093
Bump @opentelemetry/instrumentation-dns from 0.22.0 to 0.27.1 
Bumps [@opentelemetry/instrumentation-dns](https://github.com/open-telemetry/opentelemetry-js-contrib) from 0.22.0 to 0.27.1.
- [Release notes](https://github.com/open-telemetry/opentelemetry-js-contrib/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-js-contrib/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-js-contrib/compare/v0.22.0...host-metrics-v0.27.1)

---
updated-dependencies:
- dependency-name: "@opentelemetry/instrumentation-dns"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-02-14 21:21:48 +00:00
dependabot[bot]
c10cd4d831
Bump @opentelemetry/instrumentation-http from 0.22.0 to 0.27.0 
Bumps [@opentelemetry/instrumentation-http](https://github.com/open-telemetry/opentelemetry-js) from 0.22.0 to 0.27.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-js/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-js/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-js/compare/v0.22.0...experimental/v0.27.0)

---
updated-dependencies:
- dependency-name: "@opentelemetry/instrumentation-http"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-02-12 21:32:38 +00:00
Jackson Harper
8163b5a4de Upgrade axios 2022-02-11 13:51:43 -08:00
Jackson Harper
ef2b53e7f7 Upgrade Cloud Storage and the apollo-express libraries 2022-02-11 13:21:03 -08:00
Jackson Harper
84f32935f5 Open source omnivore 2022-02-11 09:24:33 -08:00