Update – 6 March 2021: PHP 8 bug fix: Warnings produced processing some sites (fixed in ContentExtractor.php). Please download version 3.9.11 from our customer portal.
Update – 2 March 2021: Improved JSON+LD extraction and fixed warnings generated with PHP 8 on some sites. Please download version 3.9.10 from our customer portal.
Update – 26 February 2021: Improved JSON-LD handling and fixed a bug with extracted JSON-LD elements not getting cleared between item fetches on a feed. Please download version 3.9.9 from our customer portal.
Update – 13 February 2021: Some users experienced problems with version 3.9.7 fetching content when running it on servers with a slightly older version of OpenSSL. If you had trouble, please download version 3.9.8 from our customer portal to fix the issue.
Full-Text RSS version 3.9.7 is now available. Full-Text RSS is used by software developers and news enthusiasts to extract article content from news sites and blogs, and to convert RSS feeds that contain only extracts of stories into full-text feeds.
Existing customers can download the latest version through our customer portal.
What’s changed in 3.9.7?
You’ll find a full changelog at the end, but here are the main changes.
PHP 8 compatible
We tested this release with PHP 8 and removed code that was deprecated.
JSON-LD aware
JSON-LD, or JavaScript Object Notation for Linked Data, is used by publishers to embed machine-readable data about articles on their websites. It’s become common practice in recent years.
We’ve noticed cases where the data that Full-Text RSS previously looked to extract from meta tags or HTML elements inside the body are, on some sites, only found inside JSON-LD objects. As such, we’ve added code to our content extractor in this version to look inside these objects for the article title, author and date.
Automatic site config updates fixed
Automatic site config updates through the Full-Text RSS admin pages stopped working recently. We rely on Github for these and the HTTP headers we looked for had changed, so we’ve now updated our code to handle it.
No longer relying on SubToMe for subscribe links
SubToMe.com offers a convenient way to subscribe to a feed in your favourite feed reader. In previous versions we used it in the feed preview and in a <atom:link rel="related"...> field in the feed output. Unfortunately it hasn’t been updated since 2015, so some of its subscribe endpoints (e.g. Feedly’s) are no longer valid.
We now point users to our own subscribe page. Here’s an example: https://subscribe.fivefilters.org/?name=FiveFilters.org&url=https%3A%2F%2Fblog.fivefilters.org%2Ffeed.xml
Note: this is currently hardcoded to use subscribe.fivefilters.org, we plan to offer this as part of the Full-Text RSS package in the future.
Installing on a VPS
If you’d like to try the new version out on a VPS, we can recommend Hetzner Cloud (free €20 credit with link if you’re new to Hetzner). Please see our installation instructions.
Feedback
Please let us know if you have any trouble using this new version.
Changelog
Full-Text RSS 3.9.10 (2021-03-02)
- Improve JSON+LD extraction and fix warnings generated with PHP 8 on some sites
- Minor improvements and API parameter description updates
Full-Text RSS 3.9.9 (2021-02-26)
- Better handling of JSON+LD elements
- Bug fix: JSON+LD extracted elements not cleared between item fetches on a feed
Full-Text RSS 3.9.8 (2021-02-13)
- Bug fix: Full-Text RSS failed to fetch content on some servers with older versions of OpenSSL.
Full-Text RSS 3.9.7 (2021-02-11)
- Bug fix: Item titles (and feed title and description) no longer double-encoded when they contain characters that need encoding
- Bug fix: Automatic site config updates (if configured via admin page) stopped working due to Github changes
- Bug fix: Query string param ‘&images=0’ to remove images from output should now work correctly
- Bug fix: Proxy servers (with auth) and PECL HTTP extension should now work correctly if configured in config file
- Look inside JSON+LD elements to extract title, author, date (use ‘skip_json_ld: yes’ in site config file to disable)
- Compatible with PHP 8
- HTML5-PHP library updated to version 2.7.4
- SimplePie library updated to version 1.5.6
- Change subtome.com URLs in the
<atom:link rel="related" href="[url]">attribute to subscribe.fivefilters.org (uses more recent feed reader subscribe endpoints) - Minor fixes