Merge remote-tracking branch 'upstream/edge' into edge

math-GH · Nov 1, 2024 · 3d1537a · 3d1537a
2 parents 348ff4b + 0735c9f
commit 3d1537a
Show file tree

Hide file tree

Showing 150 changed files with 2,301 additions and 182 deletions.
diff --git a/.typos.toml b/.typos.toml
@@ -19,6 +19,7 @@ extend-exclude = [
 	"app/i18n/el/",
 	"app/i18n/es/",
 	"app/i18n/fa/",
+	"app/i18n/fi/",
 	"app/i18n/fr/",
 	"app/i18n/he/",
 	"app/i18n/hu/",

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -5,17 +5,19 @@ See also [the FreshRSS releases](https://github.com/FreshRSS/FreshRSS/releases).
 ## 2024-1X-XX FreshRSS 1.25.0-dev
 
 * Features
-	* Add support for [regex search (regular expressions)](https://freshrss.github.io/FreshRSS/en/users/10_filter.html#regex) [#6706](https://github.com/FreshRSS/FreshRSS/pull/6706)
+	* Add support for [regex search (regular expressions)](https://freshrss.github.io/FreshRSS/en/users/10_filter.html#regex) [#6706](https://github.com/FreshRSS/FreshRSS/pull/6706), [#6926](https://github.com/FreshRSS/FreshRSS/pull/6926)
 		* ⚠️ Advanced regex syntax for searches depends on the database used (SQLite, PostgreSQL, MariaDB, MySQL),
 		but FreshRSS filter actions such as auto-mark-as-read and auto-favourite always use [PHP PCRE2 syntax](https://php.net/regexp.introduction).
 	* Allow dynamic search operator in user queries, like `search:UserQueryA date:P1d` [#6851](https://github.com/FreshRSS/FreshRSS/pull/6851)
 	* New feed mode *HTML+XPath+JSON dot notation* (JSON in HTML) [#6888](https://github.com/FreshRSS/FreshRSS/pull/6888)
 	* Better HTTP compliance with support for HTTP response headers `Cache-Control: max-age` and `Expires` [#6812](https://github.com/FreshRSS/FreshRSS/pull/6812), [FreshRSS/simplepie#26](https://github.com/FreshRSS/simplepie/pull/26)
 	* Support custom HTTP request headers per feed (e.g. for `Authorization`) [#6820](https://github.com/FreshRSS/FreshRSS/pull/6820)
 	* New unicity policies and heuristic for feeds with bad article IDs [#4487](https://github.com/FreshRSS/FreshRSS/pull/4487), [#6900](https://github.com/FreshRSS/FreshRSS/pull/6900)
+	* New option to automatically mark new articles as read if an identical title already exists in the same category [#6922](https://github.com/FreshRSS/FreshRSS/pull/6922)
 	* Add ability to remove content from articles with CSS selectors, also when not using full content [#6786](https://github.com/FreshRSS/FreshRSS/pull/6786), [#6807](https://github.com/FreshRSS/FreshRSS/pull/6807)
 	* Update `phpgt/cssxpath` library with improved CSS selectors [#6618](https://github.com/FreshRSS/FreshRSS/pull/6618)
 		* Support for `:last-child`, `:first-of-type`, `:last-of-type`, `^=`, `|=`
+	* New UI feature to download a user’ SQLite database or a database SQLite export (to be produced by CLI) [#6931](https://github.com/FreshRSS/FreshRSS/pull/6931)
 	* Better import of Inoreader user labels [#6791](https://github.com/FreshRSS/FreshRSS/pull/6791)
 	* New sharing with Telegram [#6838](https://github.com/FreshRSS/FreshRSS/pull/6838)
 * Bug fixing
@@ -46,6 +48,7 @@ See also [the FreshRSS releases](https://github.com/FreshRSS/FreshRSS/releases).
 * Security
 	* Apache protect more non-public folders and files [#6881](https://github.com/FreshRSS/FreshRSS/pull/6881), [#6893](https://github.com/FreshRSS/FreshRSS/pull/6893)
 	* Fix login in unsafe mode when using a password with special XML characters [#6797](https://github.com/FreshRSS/FreshRSS/pull/6797)
+	* Add privacy settings on extension list retrieval [#4603](https://github.com/FreshRSS/FreshRSS/pull/4603)
 * UI
 	* Searchable *My Labels* field [#6753](https://github.com/FreshRSS/FreshRSS/pull/6753)
 	* Move to next unread Label on mark as read [#6886](https://github.com/FreshRSS/FreshRSS/pull/6886)
@@ -55,6 +58,7 @@ See also [the FreshRSS releases](https://github.com/FreshRSS/FreshRSS/releases).
 	* Refactor the label menu to use a `<template>` [#6864](https://github.com/FreshRSS/FreshRSS/pull/6864)
 	* Improved subscription management page [#6816](https://github.com/FreshRSS/FreshRSS/pull/6816)
 	* Restore JavaScript form validation compatibility with Web browsers using older engines (SeaMonkey) [#6777](https://github.com/FreshRSS/FreshRSS/pull/6777)
+	* Reorganise some options [#6920](https://github.com/FreshRSS/FreshRSS/pull/6920)
 * I18n
 	* Improve German [#6847](https://github.com/FreshRSS/FreshRSS/pull/6847)
 	* Improve Italian [#6872](https://github.com/FreshRSS/FreshRSS/pull/6872)

diff --git a/CREDITS.md b/CREDITS.md
@@ -152,9 +152,11 @@ People are sorted by name so please keep this order.
 * [Mike Vanbuskirk](https://github.com/codevbus): [contributions](https://github.com/FreshRSS/FreshRSS/pulls?q=is:pr+author:codevbus) [Web](http://mikevanbuskirk.io/)
 * [miles](https://github.com/miles170): [contributions](https://github.com/FreshRSS/FreshRSS/pulls?q=is:pr+author:miles170)
 * [mincerafter42](https://github.com/mincerafter42): [contributions](https://github.com/FreshRSS/FreshRSS/pulls?q=is:pr+author:mincerafter42), [Web](https://mincerafter42.github.io)
+* [Minna N.](https://github.com/minna-xd): [contributions](https://github.com/FreshRSS/FreshRSS/pulls?q=is:pr+author:minna-xd)
 * [Mossroy](https://github.com/mossroy): [contributions](https://github.com/FreshRSS/FreshRSS/pulls?q=is:pr+author:mossroy)
 * [Mossroy](https://github.com/mossroy): [contributions](https://github.com/FreshRSS/FreshRSS/pulls?q=is:pr+author:mossroy), [Web](https://blog.mossroy.fr/)
 * [MSZ](https://github.com/mszkb): [contributions](https://github.com/FreshRSS/FreshRSS/pulls?q=is:pr+author:mszkb)
+* [mtalexan](https://github.com/mtalexan): [contributions](https://github.com/FreshRSS/FreshRSS/pulls?q=is:pr+author:mtalexan)
 * [Mubarak Harran Alketbi](https://github.com/MHketbi): [contributions](https://github.com/FreshRSS/FreshRSS/pulls?q=is:pr+author:MHketbi)
 * [Myuki](https://github.com/Myuki): [contributions](https://github.com/FreshRSS/FreshRSS/pulls?q=is:pr+author:Myuki)
 * [NaeiKinDus](https://github.com/NaeiKinDus): [contributions](https://github.com/FreshRSS/FreshRSS/pulls?q=is:pr+author:NaeiKinDus)

diff --git a/README.fr.md b/README.fr.md
@@ -234,7 +234,6 @@ et [l’API Fever](https://freshrss.github.io/FreshRSS/fr/users/06_Fever_API.htm
 | [Readrops](https://github.com/readrops/Readrops)                                      | Android     | [✔️](https://github.com/readrops/Readrops)                    | ✔️✔️                   | GReader          | ✔️            | ⭐⭐⭐       | ➖                             | ➖                  | ➖         | ➖     | ➖       | ✔️           |
 | [Fluent Reader Lite](https://hyliu.me/fluent-reader-lite/)                            | Android, iOS| [✔️](https://github.com/yang991178/fluent-reader-lite)        | ✔️✔️                   | GReader, Fever   | ✔️            | ⭐⭐⭐ | ➖                             | ➖                  | ✓         | ➖     | ➖       | ➖           |
 | [Read You](https://github.com/Ashinch/ReadYou/)                                       | Android     | [✔️](https://github.com/Ashinch/ReadYou/)                     | [En développement](https://github.com/Ashinch/ReadYou/discussions/542)        | GReader, Fever   | ➖            | ⭐⭐    | ➖                   | ✔️                   | ✔️             | ➖     | ➖       | ✔️           |
-| [ChristopheHenry](https://gitlab.com/christophehenry/freshrss-android)                | Android     | [✔️](https://gitlab.com/christophehenry/freshrss-android)     | En développement        | GReader          | ✔️            | ⭐⭐        | ➖                             | ✔️                  | ✔️         | ➖     | ➖       | ➖           |
 | [Fluent Reader](https://hyliu.me/fluent-reader/)                             | Windows, Linux, macOS| [✔️](https://github.com/yang991178/fluent-reader)             | ✔️✔️                   | Fever            | ✔️            | ⭐         | ➖                             | ✔️                  | ✓         | ➖     | ➖       | ➖           |
 | [RSS Guard](https://github.com/martinrotter/rssguard)             | Windows, GNU/Linux, macOS, OS/2 | [✔️](https://github.com/martinrotter/rssguard)                | ✔️✔️                   | GReader          | ✔️            | ⭐⭐ | ➖ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ |
 | [NewsFlash](https://gitlab.com/news-flash/news_flash_gtk)                             | GNU/Linux   | [✔️](https://gitlab.com/news-flash/news_flash_gtk)            | ✔️✔️                   | GReader, Fever   | ➖            | ⭐⭐        | ➖                           | ✔️                | ✔️       | ✔️    | ➖      | ➖          |

diff --git a/README.md b/README.md
@@ -131,7 +131,6 @@ and [Fever API](https://freshrss.github.io/FreshRSS/en/developers/06_Fever_API.h
 | [Fluent Reader Lite](https://hyliu.me/fluent-reader-lite/)                            | Android, iOS| [✔️](https://github.com/yang991178/fluent-reader-lite)        | ✔️✔️                   | GReader, Fever   | ✔️            | ⭐⭐⭐    | ➖                             | ➖                  | ✓         | ➖     | ➖       | ➖           |
 | [FocusReader](https://play.google.com/store/apps/details?id=allen.town.focus.reader)  | Android     | ➖                                                            | ✔️✔️                   | GReader          | ✔️            | ⭐⭐⭐    | ➖                             | ➖                  | ✔️         | ➖     | ✓       | ✔️           |
 | [Read You](https://github.com/Ashinch/ReadYou/)                                       | Android     | [✔️](https://github.com/Ashinch/ReadYou/)                     | [Work in progress](https://github.com/Ashinch/ReadYou/discussions/542)        | GReader, Fever   | ➖            | ⭐⭐     | ➖                    | ✔️                   | ✔️             | ➖     | ➖       | ✔️           |
-| [ChristopheHenry](https://gitlab.com/christophehenry/freshrss-android)                | Android     | [✔️](https://gitlab.com/christophehenry/freshrss-android)     | Work in progress        | GReader          | ✔️            | ⭐⭐      | ➖                             | ✔️                  | ✔️         | ➖     | ➖       | ➖           |
 | [Fluent Reader](https://hyliu.me/fluent-reader/)                             | Windows, Linux, macOS| [✔️](https://github.com/yang991178/fluent-reader)             | ✔️✔️                   | GReader, Fever   | ✔️            | ⭐        | ➖                             | ✔️                  | ✓         | ➖     | ➖       | ➖           |
 | [RSS Guard](https://github.com/martinrotter/rssguard)             | Windows, GNU/Linux, macOS, OS/2 | [✔️](https://github.com/martinrotter/rssguard)                | ✔️✔️                   | GReader          | ✔️            | ⭐⭐      | ➖ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ |
 | [NewsFlash](https://gitlab.com/news-flash/news_flash_gtk)                             | GNU/Linux   | [✔️](https://gitlab.com/news-flash/news_flash_gtk)            | ✔️✔️                   | GReader, Fever | ➖            | ⭐⭐      | ➖                           | ✔️                | ✔️       | ✔️    | ➖      | ➖          |

diff --git a/app/Controllers/categoryController.php b/app/Controllers/categoryController.php
@@ -100,6 +100,12 @@ public function updateAction(): void {
 		FreshRSS_View::prependTitle($category->name() . ' · ' . _t('sub.title') . ' · ');
 
 		if (Minz_Request::isPost()) {
+			if (Minz_Request::paramBoolean('enable_read_when_same_title_in_category')) {
+				$category->_attribute('read_when_same_title_in_category', Minz_Request::paramInt('read_when_same_title_in_category'));
+			} else {
+				$category->_attribute('read_when_same_title_in_category', null);
+			}
+
 			$category->_filtersAction('read', Minz_Request::paramTextToArray('filteractions_read'));
 
 			if (Minz_Request::paramBoolean('use_default_purge_options')) {

diff --git a/app/Controllers/configureController.php b/app/Controllers/configureController.php
@@ -503,4 +503,20 @@ public function systemAction(): void {
 			Minz_Request::good(_t('feedback.conf.updated'), [ 'c' => 'configure', 'a' => 'system' ]);
 		}
 	}
+
+	public function privacyAction(): void {
+		if (!FreshRSS_Auth::hasAccess('admin')) {
+			Minz_Error::error(403);
+		}
+
+		if (Minz_Request::isPost()) {
+			FreshRSS_Context::userConf()->retrieve_extension_list = Minz_Request::paramBoolean('retrieve_extension_list');
+			FreshRSS_Context::userConf()->save();
+			invalidateHttpCache();
+
+			Minz_Request::good(_t('feedback.conf.updated'), array('c' => 'configure', 'a' => 'privacy'));
+		}
+
+		FreshRSS_View::prependTitle(_t('conf.privacy') . ' · ');
+	}
 }
diff --git a/app/Controllers/extensionController.php b/app/Controllers/extensionController.php
@@ -44,7 +44,18 @@ public function indexAction(): void {
 	 */
 	protected function getAvailableExtensionList(): array {
 		$extensionListUrl = 'https://raw.githubusercontent.com/FreshRSS/Extensions/master/extensions.json';
-		$json = httpGet($extensionListUrl, CACHE_PATH . '/extension_list.json', 'json');
+
+		$cacheFile = CACHE_PATH . '/extension_list.json';
+		if (FreshRSS_Context::userConf()->retrieve_extension_list === true) {
+			if (!file_exists($cacheFile) || (time() - (filemtime($cacheFile) ?: 0) > 86400)) {
+				$json = httpGet($extensionListUrl, $cacheFile, 'json');
+			} else {
+				$json = @file_get_contents($cacheFile) ?: '';
+			}
+		} else {
+			Minz_Log::warning('The extension list retrieval is disabled in privacy configuration');
+			return [];
+		}
 
 		// we ran into problems, simply ignore them
 		if ($json === '') {

diff --git a/app/Controllers/feedController.php b/app/Controllers/feedController.php
@@ -409,6 +409,7 @@ public static function actualizeFeeds(?int $feed_id = null, ?string $feed_url =
 			$maxFeeds = PHP_INT_MAX;
 		}
 
+		$catDAO = FreshRSS_Factory::createCategoryDao();
 		$feedDAO = FreshRSS_Factory::createFeedDao();
 		$entryDAO = FreshRSS_Factory::createEntryDao();
 
@@ -425,7 +426,6 @@ public static function actualizeFeeds(?int $feed_id = null, ?string $feed_url =
 
 			// Hydrate category for each feed to avoid that each feed has to make an SQL request
 			$categories = [];
-			$catDAO = FreshRSS_Factory::createCategoryDao();
 			foreach ($catDAO->listCategories(false, false) as $category) {
 				$categories[$category->id()] = $category;
 			}
@@ -444,6 +444,8 @@ public static function actualizeFeeds(?int $feed_id = null, ?string $feed_url =
 		$nbUpdatedFeeds = 0;
 		$nbNewArticles = 0;
 		$feedsCacheToRefresh = [];
+		/** @var array<int,array<string,true>> */
+		$categoriesEntriesTitle = [];
 
 		foreach ($feeds as $feed) {
 			$feed = Minz_ExtensionManager::callHook('feed_before_actualize', $feed);
@@ -563,6 +565,14 @@ public static function actualizeFeeds(?int $feed_id = null, ?string $feed_url =
 					$titlesAsRead = [];
 				}
 
+				$category = $feed->category();
+				if (!isset($categoriesEntriesTitle[$feed->categoryId()]) && $category !== null && $category->hasAttribute('read_when_same_title_in_category')) {
+					$categoriesEntriesTitle[$feed->categoryId()] = array_fill_keys(
+						$catDAO->listTitles($feed->categoryId(), $category->attributeInt('read_when_same_title_in_category') ?? 0),
+						true
+					);
+				}
+
 				$mark_updated_article_unread = $feed->attributeBoolean('mark_updated_article_unread') ?? FreshRSS_Context::userConf()->mark_updated_article_unread;
 
 				// For this feed, check existing GUIDs already in database.
@@ -603,6 +613,9 @@ public static function actualizeFeeds(?int $feed_id = null, ?string $feed_url =
 							if ($readWhenSameTitleInFeed > 0) {
 								$titlesAsRead[$entry->title()] = true;
 							}
+							if (isset($categoriesEntriesTitle[$feed->categoryId()])) {
+								$categoriesEntriesTitle[$feed->categoryId()][$entry->title()] = true;
+							}
 
 							if (!$entry->isRead()) {
 								$needFeedCacheRefresh = true;	//Maybe
@@ -625,10 +638,13 @@ public static function actualizeFeeds(?int $feed_id = null, ?string $feed_url =
 							continue;
 						}
 
-						$entry->applyFilterActions($titlesAsRead);
+						$entry->applyFilterActions(array_merge($titlesAsRead, $categoriesEntriesTitle[$feed->categoryId()] ?? []));
 						if ($readWhenSameTitleInFeed > 0) {
 							$titlesAsRead[$entry->title()] = true;
 						}
+						if (isset($categoriesEntriesTitle[$feed->categoryId()])) {
+							$categoriesEntriesTitle[$feed->categoryId()][$entry->title()] = true;
+						}
 
 						$needFeedCacheRefresh = true;
 

diff --git a/app/Controllers/importExportController.php b/app/Controllers/importExportController.php
@@ -31,6 +31,7 @@ public function firstAction(): void {
 	public function indexAction(): void {
 		$this->view->feeds = $this->feedDAO->listFeeds();
 		FreshRSS_View::prependTitle(_t('sub.import_export.title') . ' · ');
+		$this->listSqliteArchives();
 	}
 
 	private static function megabytes(string $size_str): float|int|string {
@@ -694,4 +695,41 @@ private static function filenameToContentType(string $filename): string {
 				return 'application/octet-stream';
 		}
 	}
+
+	private const REGEX_SQLITE_FILENAME = '/^(?![.-])[0-9a-zA-Z_.@ #&()~\-]{1,128}\.sqlite$/';
+
+	private function listSqliteArchives(): void {
+		$this->view->sqliteArchives = [];
+		$files = glob(USERS_PATH . '/' . Minz_User::name() . '/*.sqlite', GLOB_NOSORT) ?: [];
+		foreach ($files as $file) {
+			$archive = [
+				'name' => basename($file),
+				'size' => @filesize($file),
+				'mtime' => @filemtime($file),
+			];
+			if ($archive['size'] != false && $archive['mtime'] != false && preg_match(self::REGEX_SQLITE_FILENAME, $archive['name'])) {
+				$this->view->sqliteArchives[] = $archive;
+			}
+		}
+		// Sort by time, newest first:
+		usort($this->view->sqliteArchives, static fn(array $a, array $b): int => $b['mtime'] <=> $a['mtime']);
+	}
+
+	public function sqliteAction(): void {
+		if (!Minz_Request::isPost()) {
+			Minz_Request::forward(['c' => 'importExport', 'a' => 'index'], true);
+		}
+		$sqlite = Minz_Request::paramString('sqlite');
+		if (!preg_match(self::REGEX_SQLITE_FILENAME, $sqlite)) {
+			Minz_Error::error(404);
+			return;
+		}
+		$path = USERS_PATH . '/' . Minz_User::name() . '/' . $sqlite;
+		if (!file_exists($path) || @filesize($path) == false || @filemtime($path) == false) {
+			Minz_Error::error(404);
+			return;
+		}
+		$this->view->sqlitePath = $path;
+		$this->view->_layout(null);
+	}
 }
diff --git a/app/Models/BooleanSearch.php b/app/Models/BooleanSearch.php
@@ -35,6 +35,7 @@ public function __construct(string $input, int $level = 0, string $operator = 'A
 		$this->raw_input = $input;
 
 		if ($level === 0) {
+			$input = self::escapeRegexParentheses($input);
 			$input = $this->parseUserQueryNames($input, $allowUserQueries);
 			$input = $this->parseUserQueryIds($input, $allowUserQueries);
 			$input = trim($input);
@@ -78,7 +79,7 @@ private function parseUserQueryNames(string $input, bool $allowUserQueries = tru
 					if (!empty($queries[$name])) {
 						$fromS[] = $matches[0][$i];
 						if ($allowUserQueries) {
-							$toS[] = '(' . $queries[$name] . ')';
+							$toS[] = '(' . self::escapeRegexParentheses($queries[$name]) . ')';
 						} else {
 							$toS[] = '';
 						}
@@ -119,7 +120,7 @@ private function parseUserQueryIds(string $input, bool $allowUserQueries = true)
 					if (!empty($queries[$id])) {
 						$fromS[] = $matches[0][$i];
 						if ($allowUserQueries) {
-							$toS[] = '(' . $queries[$id] . ')';
+							$toS[] = '(' . self::escapeRegexParentheses($queries[$id]) . ')';
 						} else {
 							$toS[] = '';
 						}
@@ -132,6 +133,20 @@ private function parseUserQueryIds(string $input, bool $allowUserQueries = true)
 		return $input;
 	}
 
+	/**
+	 * Temporarily escape parentheses used in regex expressions.
+	 */
+	public static function escapeRegexParentheses(string $input): string {
+		return preg_replace_callback('#(?<=[\\s(:!-]|^)(?<![\\\\])/.+?(?<!\\\\)/[im]*#',
+			fn(array $matches): string => str_replace(['(', ')'], ['\\u0028', '\\u0029'], $matches[0]),
+			$input
+		) ?? '';
+	}
+
+	public static function unescapeRegexParentheses(string $input): string {
+		return str_replace(['\\u0028', '\\u0029'], ['(', ')'], $input);
+	}
+
 	/**
 	 * Example: 'ab cd OR ef OR "gh ij"' becomes '(ab cd) OR (ef) OR ("gh ij")'
 	 */