Commit graph

5269 commits

Author SHA1 Message Date
Collin M. Barrett
a101dcaad2 refactor(services): ♻ exclude pdf from prod builds 2020-09-20 10:20:03 -05:00
Collin M. Barrett
e416f96fcb fix(archival): 🐛 don't trim mscorlib 2020-09-20 10:10:08 -05:00
Collin M. Barrett
c8834369b4 Revert "fix(archival): 🐛 temp disable trimming to try resolving FileNotFoundException"
This reverts commit 4c875e7f3b.
2020-09-20 09:59:56 -05:00
Collin M. Barrett
4c875e7f3b fix(archival): 🐛 temp disable trimming to try resolving FileNotFoundException 2020-09-20 08:19:57 -05:00
Collin M. Barrett
354c95acad feat(archival): log archive job metrics in DEBUG and RELEASE 2020-09-19 18:33:05 -05:00
Collin M. Barrett
bb14e212e9 docs(archival): 📝 add release badge to readme 2020-09-19 18:09:02 -05:00
Collin M. Barrett
08725dd99a fix(archival): 🐛 don't re-create directory when archival is released 2020-09-19 17:57:06 -05:00
Collin M. Barrett
6c9ae48819 feat(archival): catch and log HttpRequestException 2020-09-19 17:23:03 -05:00
Collin M. Barrett
2f187cd9da feat(archival): ♻ store txt files in configured dir, ren many members 2020-09-19 17:13:52 -05:00
Collin M. Barrett
501cad0ee8 docs(archival): 📝 expand TODO 2020-09-19 12:47:49 -05:00
Collin M. Barrett
79421a443a feat(archival): log enqueuing of archive job stats 2020-09-19 12:46:37 -05:00
Collin M. Barrett
b1ba4dfb3d refactor(archival): ♻ 'hope' that multiple jobs don't conflict between git stage and commit 2020-09-19 12:39:55 -05:00
Collin M. Barrett
09f35b2922 docs(archival): 📝♻ mv TODO 2020-09-19 10:26:12 -05:00
Collin M. Barrett
1ed13d4e9f docs(archival): 📝 add TODO 2020-09-19 10:12:48 -05:00
Collin M. Barrett
7f0ee525a2 feat(archival): equally space archival jobs throughout the day to distribute resources 2020-09-19 10:05:44 -05:00
Collin M. Barrett
eed80de6d2 feat(archival): support scheduling mediatR cmds w/delay 2020-09-19 09:53:21 -05:00
Collin M. Barrett
ffc7df65f6 fix(archival): 🐛 fix url filtering by segmentNumber/primariness 2020-09-19 09:46:22 -05:00
Collin M. Barrett
12443645c5 fix(archival): 🐛 fix log statement param 2020-09-19 09:30:50 -05:00
Collin M. Barrett
18444c0117 refactor(archival): ♻ make Hangfire using conditional 2020-09-19 09:28:26 -05:00
Collin M. Barrett
0d1a7a2f96 refactor(reverse-proxy): ♻ debug reverse-proxy on 8080 2020-09-19 09:25:43 -05:00
Collin M. Barrett
cee379cf9f feat(archival): in DEBUG, archive only 25 lists and on startup 2020-09-19 09:23:08 -05:00
Collin M. Barrett
92cc505faa refactor(archival): ♻ inline local 2020-09-19 09:16:30 -05:00
Collin M. Barrett
e4d9aae065 fix(archival): 🐛 don't write file if no stream conversion strategy exists 2020-09-19 07:03:29 -05:00
Collin M. Barrett
7f37cb15df feat(archival): archive lists in random order 2020-09-19 06:56:27 -05:00
Collin M. Barrett
6dd066f82d feat(archival): left pad archive filenames for better sorting 2020-09-19 06:55:43 -05:00
Azure Pipelines EF Migration Bot
f65f7cb359 feat(data): migrate PR #1997 2020-09-19 10:39:57 +00:00
MDLeom
f5c65c11f6 filter(phishing-filter): add adguard & vivaldi-compatible blocklists 2020-09-19 10:39:57 +00:00
Collin M. Barrett
999fe0595a Merge branch 'master' of https://github.com/collinbarrett/FilterLists 2020-09-18 16:57:35 -05:00
Collin M. Barrett
e95892af1b feat(archival): ♻ support segments of different source types, use IAsyncEnumerable for Streams 2020-09-18 16:57:28 -05:00
dependabot[bot]
623f5f7661 build(deps): bump @types/node from 14.10.3 to 14.11.1 in /web
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 14.10.3 to 14.11.1.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

Signed-off-by: dependabot[bot] <support@github.com>
2020-09-18 12:24:26 +00:00
Collin M. Barrett
c4d97dd8b4 feat(archival): normalize target filenames 2020-09-17 20:54:37 -05:00
Collin M. Barrett
218b9ac268 feat(archival): add retry to new typed FileClient 2020-09-17 19:26:35 -05:00
Collin M. Barrett
1f916ccb58 feat(api-clients): add retry to IDirectoryClient transient errors 2020-09-17 18:08:25 -05:00
Collin M. Barrett
9c367dbfc6 docs(archival): 📝 fix todo typo 2020-09-17 17:30:08 -05:00
Collin M. Barrett
716cd38284 docs(archival): 📝 add Polly todos 2020-09-17 17:28:41 -05:00
Collin M. Barrett
a4844c7679 refactor(archival): 🎨 prefer fileName/FileName to filename 2020-09-17 17:19:19 -05:00
Collin M. Barrett
0e1f4b791f fix(archival): 🐛📝 finish writing Stream before disposing HttpResponseMessage, add TODOs 2020-09-17 17:14:47 -05:00
Collin M. Barrett
66cb92eabc fix(archival): 🐛 buffer stream to memory so HttpClient doesn't dispose before read completes 2020-09-17 15:53:23 -05:00
Collin M. Barrett
67d38b1b15 Merge branch 'master' of https://github.com/collinbarrett/FilterLists 2020-09-17 09:34:58 -05:00
Collin M. Barrett
6b377e7d17 docs(archival): 📝 add TODO 2020-09-17 09:34:51 -05:00
Azure Pipelines EF Migration Bot
5b5978f26d feat(data): migrate PR #1992 2020-09-17 12:29:13 +00:00
Collin M. Barrett
1b557c892b fix(directory): 🐛 add app insights directory to appsettings.*.json to unbreak migrations 2020-09-17 12:29:13 +00:00
MDLeom
ee1ccb50fb filter(urlhaus-filter): dedup filters
- Remove blocklists with offline links
2020-09-17 12:29:13 +00:00
MDLeom
14deb36c9f filter(urlhaus-filter): add adguard & vivaldi-compatible blocklists 2020-09-17 12:29:13 +00:00
dependabot[bot]
27f21ca053 build(deps): bump @types/jest from 26.0.13 to 26.0.14 in /web
Bumps [@types/jest](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/jest) from 26.0.13 to 26.0.14.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/jest)

Signed-off-by: dependabot[bot] <support@github.com>
2020-09-17 11:53:32 +00:00
Collin M. Barrett
8166dec254 feat(archival): implement strategy pattern for file writing 2020-09-17 06:41:36 -05:00
dependabot[bot]
478b90a806 build(deps): bump @types/node from 14.10.2 to 14.10.3 in /web
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 14.10.2 to 14.10.3.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

Signed-off-by: dependabot[bot] <support@github.com>
2020-09-17 11:39:07 +00:00
dependabot[bot]
03c8350ac8 build(deps-dev): bump prettier from 2.1.1 to 2.1.2 in /web
Bumps [prettier](https://github.com/prettier/prettier) from 2.1.1 to 2.1.2.
- [Release notes](https://github.com/prettier/prettier/releases)
- [Changelog](https://github.com/prettier/prettier/blob/master/CHANGELOG.md)
- [Commits](https://github.com/prettier/prettier/compare/2.1.1...2.1.2)

Signed-off-by: dependabot[bot] <support@github.com>
2020-09-16 11:09:52 +00:00
dependabot[bot]
7fd127908a build(deps): bump Swashbuckle.AspNetCore
Bumps [Swashbuckle.AspNetCore](https://github.com/domaindrivendev/Swashbuckle.AspNetCore) from 5.6.0 to 5.6.1.
- [Release notes](https://github.com/domaindrivendev/Swashbuckle.AspNetCore/releases)
- [Commits](https://github.com/domaindrivendev/Swashbuckle.AspNetCore/compare/v5.6.0...v5.6.1)

Signed-off-by: dependabot[bot] <support@github.com>
2020-09-16 11:09:36 +00:00
Collin M. Barrett
135809969a refactor(archival): ♻ ren IFileArchiver 2020-09-15 19:59:13 -05:00