7989 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Hongbo Wu	eb0f0b6c4f	improve data migration speed	2023-10-05 14:32:05 +08:00
Hongbo Wu	86478c13d2	fix sql	2023-10-05 14:32:05 +08:00
Hongbo Wu	42e01613d4	inner join user table when inserting highlight to make sure user exists	2023-10-05 14:32:05 +08:00
Hongbo Wu	cbf58614c4	use log levels in custom typeorm logger	2023-10-05 14:32:05 +08:00
Hongbo Wu	efdbc4f345	use interpolated string or dictionary parameter in logs	2023-10-05 14:32:03 +08:00
Hongbo Wu	4d846c520b	create a new sanitized tuple because tuple is immutable	2023-10-05 14:31:07 +08:00
Hongbo Wu	475e32e859	Catch error from converting string to datetime	2023-10-05 14:31:07 +08:00
Hongbo Wu	71c203cdb2	Update migration script to check the unique index while inserting data into library_item table	2023-10-05 14:31:07 +08:00
Hongbo Wu	e950662ced	Replace the unique constraint on library_item table with a unique index of userId and md5 hashed original url	2023-10-05 14:31:07 +08:00
Hongbo Wu	6604c42bf4	remove null byte from content	2023-10-05 14:31:07 +08:00
Hongbo Wu	093440e1ab	replace surrogates with placeholder	2023-10-05 14:31:07 +08:00
Hongbo Wu	aa9c97fcbc	encode with utf-16 if failed	2023-10-05 14:31:07 +08:00
Hongbo Wu	f9ce8f6b3e	encode surrogatepass to avoid error when inserting into postgres	2023-10-05 14:31:07 +08:00
Hongbo Wu	65115b3ba7	fix a typo	2023-10-05 14:31:07 +08:00
Hongbo Wu	4c30a4f561	use enumerate to get index from for loop	2023-10-05 14:31:07 +08:00
Hongbo Wu	e68f7b42ba	print original id for debugging	2023-10-05 14:31:07 +08:00
Hongbo Wu	71e3c716be	excute insert query one by one if batch insert failed and if string is too long for tsvector, disable the update_library_item_tsv trigger temporarily and insert the failed one again	2023-10-05 14:31:07 +08:00
Hongbo Wu	663ccdae41	try to insert library item one by one if batch insert failed	2023-10-05 14:31:07 +08:00
Hongbo Wu	323be5376a	truncate long description to 1MB	2023-10-05 14:31:07 +08:00
Hongbo Wu	85febfa8b6	fix a typo	2023-10-05 14:31:07 +08:00
Hongbo Wu	c7e56fbce6	add PG_TIMEOUT to env var	2023-10-05 14:31:07 +08:00
Hongbo Wu	6d6d34e94e	convert ES_TIMEOUT to int	2023-10-05 14:31:07 +08:00
Hongbo Wu	e9527bb5ad	add es_timeout to env var	2023-10-05 14:31:07 +08:00
Hongbo Wu	f46006e293	add end_time to the migrate script	2023-10-05 14:31:07 +08:00
Hongbo Wu	4d1d9a8c1b	skip item if content length is greater than 1MB	2023-10-05 14:31:07 +08:00
Hongbo Wu	327f6959ef	fix highlight createdByMe is wrong	2023-10-05 14:31:07 +08:00
Hongbo Wu	0ff9fdef04	add elastic scroll time in the env var	2023-10-05 14:31:07 +08:00
Hongbo Wu	0912e7cf0f	remove warn logs	2023-10-05 14:31:07 +08:00
Hongbo Wu	3b8b48bc21	fix recovered deleted items still shown in trash	2023-10-05 14:31:07 +08:00
Hongbo Wu	7bf3f1bbf4	set deletedAt to now when bulk delete	2023-10-05 14:31:07 +08:00
Hongbo Wu	47510c0785	fix recommendations not fetched in search results	2023-10-05 14:31:07 +08:00
Hongbo Wu	9c723ecbd3	fix id not set in page update pubsub message	2023-10-05 14:31:07 +08:00
Hongbo Wu	0c691257eb	fix rules permission	2023-10-05 14:31:07 +08:00
Hongbo Wu	5f6587537e	fix recommendation wildcard search	2023-10-05 14:31:07 +08:00
Hongbo Wu	bf2c54084b	fix getArticle API failed on iOS	2023-10-05 14:31:07 +08:00
Hongbo Wu	9e5196406e	revert rls on old tables	2023-10-05 14:31:07 +08:00
Hongbo Wu	d5c247a98d	make cool-down time an env var	2023-10-05 14:31:07 +08:00
Hongbo Wu	330e50fba5	increase scroll to 2m and cool down for 5 seconds	2023-10-05 14:31:07 +08:00
Hongbo Wu	7d4948a43b	reduce scroll to 1m	2023-10-05 14:31:07 +08:00
Hongbo Wu	8f1664b058	increase timeout	2023-10-05 14:31:07 +08:00
Hongbo Wu	a83f7921f4	reduce shortId length to 14	2023-10-05 14:31:07 +08:00
Hongbo Wu	a947310d30	reduce shortId length to 14	2023-10-05 14:31:07 +08:00
Hongbo Wu	ecbabacc78	set reading progress to 0 if null	2023-10-05 14:31:07 +08:00
Hongbo Wu	a445bc55d4	add default item_type	2023-10-05 14:31:07 +08:00
Hongbo Wu	b4512d2d1c	convert ES_SCAN_SIZE to int	2023-10-05 14:31:07 +08:00
Hongbo Wu	b20ad4a210	reorder migrate sql	2023-10-05 14:31:07 +08:00
Hongbo Wu	2e5529fd81	allow readingProgressAnchorIndex to be null	2023-10-05 14:31:07 +08:00
Hongbo Wu	8712107a2f	remove db caching	2023-10-05 14:31:07 +08:00
Hongbo Wu	f3db9345ae	add test for when in:trash is:unread is in the query	2023-10-05 14:31:07 +08:00
Hongbo Wu	c57ead3583	add test for when rss:feed in:archive is in the query	2023-10-05 14:31:07 +08:00