Home
This wiki covers syndication as a system.
For how to install and run each individual part of the system, you should look at the README.md file in the relevant repo.
Syndication (also known as Republishing), is the term for allowing other publishers to use our content in their own publications.
Once the user has syndication licence, ft.com appears sightly different.
When on ft.com, users see coloured symbols next to the title of every article/podcast/video on the page.
The symbols indicates different syndication grants:
On clicking on the green tick icon, the user can save or download the content
On the top right corner on the page, the Republishing link takes users to the republishing platform.
On the republishing tool, Users can see their contract details, allowance details and select their preferred download format.
Users can also manage saved, downloaded and Spanish content.
This is the client side library responsible for adding syndication symbols next to article/podcast and video headlines on ft.com. Allowing syndication customers to download or save content for republishing.
What it does
- Checks if logged in user is a syndication user
- Gets all the
contentIdsfrom the DOM
- Passes them to next-syndication-api to get the syndication data for each content ID
- Uses the response to put the syndication symbols
This is the application for the Republishing platform.
This is the REST API responsible for powering Syndication. It lives in Heroku with a Postgres DB attached. It handles requests from n-syndication and next-syn-list. It also runs some Cron jobs and sync tasks.
Here are the available API endpoints
What it does
- Manages connection and interactions with attached Postgres DB
- Stores most of its data in the Postgres DB
- Periodically updates information about contracts with up to date data from salesforce.
- Data about saved and downloaded content is added to the DB
- It stores Personal Identifiable Information (Membership API were flaky)
Cron Jobs
-
Backup
- Runs periodically.
- Export database schema, zips it and uploads it to the
productionfolder of thenext-syndication-db-backupsbucket on AWS (FT Infra Prod account).
-
Redshift
- Runs periodically.
- Export analytics and uploads it to the
redshiftfolder of thenext-syndication-db-backupsbucket on AWS (FT Infra Prod account).
-
Tidy-Bucket
- Runs periodically.
- Deletes backup and redshift files older than one month from the S3 bucket.
The cron jobs are defined here
Sync Tasks
-
content-es
- Subscribes to
next-syndication-translationsSQS on AWS(FT Infra Prod account). - When a
createdevent is triggered, the content is created / updated in the PostgresDB . - When a
deletedevent is triggered, the content is marked as deleted in the PostgresDB.
- Subscribes to
-
db-persist
- Subscribes to
next-syndication-downloads-prodSQS on AWS(FT Infra Prod account). - When save, unsave, and download event is triggered, the event is
- Written into the PostgresDB
- Published to Spoor
- For download event that requires contributor payment, an email is sent to [email protected] containing content, user and contract data.
- Subscribes to
The sync tasks are defined here
Deploys next-syndication-api for the purposes of running the downloads as a separate application.
- Serverless application
- One function - transformXml
- Used for Spanish content
How does it work - Spanish Content
- A 3rd Party, Vanguard Publications translates the articles
- They upload the XML file into an S3 bucket called ft-article-translations-en-to-es-from-vanguard-publications
- This triggers a Lambda function which transforms the XML into JSON and puts it in another S3 bucket called ft-next-content-translations
- That triggers an event (on create and delete), which prompts the syndication-api to go and fetch the content, and upserts it into the Postgres DB (syndication.content_es)
Syndication - Cron Jobs | Lucidchart
Syndication - Sync Tasks | Lucidchart
