Commit graph

36 commits

Author SHA1 Message Date
Hongbo Wu
35ca7ede82 Add test case for danluu 2022-06-06 22:37:05 +08:00
Jackson Harper
704726dc6a Improve parsing of channel news asia 2022-06-03 14:03:14 -07:00
Hongbo Wu
304fe70113 Fix tests 2022-06-01 10:30:23 +08:00
Hongbo Wu
98ecdcff80 Fix test case of setting tweet-placeholder 2022-05-31 22:51:27 +08:00
Hongbo Wu
404805e0c0 Make async calls to parse() 2022-05-31 22:51:00 +08:00
Hongbo Wu
417ed0a4eb Fetch tweet id from url 2022-05-31 20:02:54 +08:00
Hongbo Wu
b6fef171be If we have a node with only one child element which has the placeholder class, keep it 2022-05-31 14:18:39 +08:00
Hongbo Wu
22f5e1cc32 Fix embed tweets got deleted when simplifying nested elements 2022-05-31 13:42:16 +08:00
Hongbo Wu
cc5bdf96f5 Update electrek test case 2022-05-31 12:52:33 +08:00
Hongbo Wu
bce50c2a92 Update electrek test case 2022-05-31 11:11:06 +08:00
Hongbo Wu
9dc7fd4c4c Add test case for electrek 2022-05-30 22:47:47 +08:00
Hongbo Wu
35dcd00ec3 Add test case for github blog 2022-05-18 15:52:47 +08:00
Hongbo Wu
0e4cec5e25 Update tests 2022-05-18 11:28:47 +08:00
Jackson Harper
6795508942 Add test for readability on youtube embeds 2022-05-18 09:55:36 +08:00
Jackson Harper
bdfa76d716 Remove listnav elements from articles 2022-05-12 15:58:24 -07:00
Hongbo Wu
d542d31aed Fix gflownet test generation 2022-05-11 21:23:30 +08:00
Hongbo Wu
82fb8151a4 Fix generate tests 2022-05-10 21:10:20 +08:00
Hongbo Wu
96b543946d Temporarily disable customer content serializer test 2022-05-10 21:10:06 +08:00
Hongbo Wu
ffa5dee721 Use linkedom in readability tests 2022-05-10 18:40:52 +08:00
Hongbo Wu
cb7f30607a Use linkedom in readability test isProbablyReaderable 2022-05-10 18:33:29 +08:00
Jackson Harper
6ef14e1f91 Mark ft.com page a readerable 2022-05-05 09:28:22 -07:00
Jackson Harper
384c5dbf9f Improve rendering of the Financial Times 2022-05-05 09:14:51 -07:00
Jackson Harper
74693d40c0 Mark page as readerable 2022-05-04 15:01:35 -07:00
Jackson Harper
8386aebaf8 Remove ads from fiercepharma page 2022-05-04 14:22:08 -07:00
Jackson Harper
a24b976546 Remove lazy loaded srcset elements
Some tools like jetpack: https://jetpack.com/support/lazy-images/
use a temporary srcset element set to a data image when lazy
loading, these are later removed by JS. We test if there is
a valid src attribute and if the srcset attribute is a data embed
to remove these.
2022-04-29 10:05:33 -07:00
Jackson Harper
d2bb359f5c Handle srcset image density specifications 2022-04-26 15:18:23 -07:00
Hongbo Wu
72a231c97e
Do not proxy image data uri (#421)
* do not proxy image data uri

* rename data uri

* rename data uri in test
2022-04-14 12:59:12 +08:00
Hongbo Wu
4d01f689b2 replace tables of article content with divs for newsletters 2022-04-11 20:00:11 +08:00
Hongbo Wu
6d405432af
add site_name and site_icon to page model and return in resolver (#341)
* add site_name and site_icon to page model and return in resolver

* fix tests
2022-03-30 10:43:10 +08:00
Jackson Harper
12af64609c Fix readability issues with null style elements
isProbably visible can fail in this case because style could be
undefined on an element.
2022-03-23 13:35:00 -07:00
Hongbo Wu
960a22d50c
Fix/city journal parsing (#266)
* remove arrow image when parsing

* ignore m_article classname element which indicates a mobile version of the website

* generate test page for city journal
2022-03-21 22:53:21 +08:00
Jackson Harper
0361ef86fa Better handling of HTML entities in descriptions
The HTML code method didnt implent all possible
entities, causing some (usually rquote) to display.
2022-03-14 11:02:08 -07:00
Jackson Harper
fc7d972855 Fix typo in readability date handling causing this parse issue
Can remove our special handler for the published date now that we
are pulling it out correctly.
2022-03-14 10:20:19 -07:00
Jackson Harper
8a2bb0f49d Handle blogger sites that display the full feed on the article page 2022-03-10 13:48:39 -08:00
Jackson Harper
234dba9174 Improve readability of channelnewsasia
This uses negative lookahead to reject nodes that have outstream
ads embedded. Previously they were being accepted because they
contained `$article` in the class name.
2022-03-04 10:34:44 -08:00
Jackson Harper
84f32935f5 Open source omnivore 2022-02-11 09:24:33 -08:00