Skip to content

API: Content Recommendations

Overview

secret parameter not required

The endpoints on this page are designed to be used in-browser (called via JavaScript), so no API secret is necessary.

Related content rather than high click through rates

At a high level, our system optimizes showing similar content rather than high click through rates. We chose to take this approach because systems that optimize for click-through rate tend to promote click-bait. You may be familiar with recommendation blocks below articles that display outrageous images or feature headlines like The ten most unexpected mistakes that could kill you today. This sort of content rises to the top because the system that chooses recommendations focuses narrowly on attracting clicks. Parse.ly’s recommendation system takes a different approach by answering the following question: if the reader wants to learn more about the topics covered by the content they are currently reading, what should we recommend?

Note: if you’re interested in displaying content that has received the most traffic recently, you can use our analytics endpoint or our top endpoint.

How do Parse.ly Recommendations work?

When you become a Parse.ly customer, we crawl all of your historical content and continue to crawl all new pieces of content in real time. Each time we crawl a piece of content, we analyze the title and full content to create an embedding and store that in a database. An embedding is a numerical representation of content in a special, high-dimensional space where documents that contain similar meaning are located closely together.

When the Parse.ly API is asked to provide a recommendation for a given document, we simply search through this embedding space for that document’s neighbors. The ordering of the recommended documents simply reflects how close they are to the query document in the embedding-space. (As we’ll see below, it’s possible to re-order these documents to boost recent pieces of content in the ranking).

How can I filter or tweak the recommendations?

If you notice a problem with the content recommended by the Parse.ly API, you might be able to use filters, boosting, or exclusion to improve the results.

Filtering

Using filters, you can choose to only recommend content that’s been

  • Assigned to a particular section
  • Given a particular tag
  • Written by a particular author
  • Published in a given publication date range

Note: a request to our API can have multiple filters attached to it. See the docs for more information.

Boosting

Boosting is useful if you want to nudge the API to bump up certain types of articles in the ranking, but don’t want to make any hard rules. For example, many customers would prefer to recommend related content that’s been recently published. In this case, the request to our api should include the `strategy` query parameter and set its value to be `recency`.  

Excluding content

If you want to block certain pieces of content from showing up in the recommendations, you can do so by using our /exclude endpoint.  Note that excluded pieces of content will also not show up in any of our API’s endpoints (such as the /analytics or /top endpoints).

What about personalization?

Our approach towards personalized closely resembles our basic approach to content recommendations.  The basic approach, which we have outlined above, is to serve content that’s similar to the currently viewed piece of content.  In the case of personalization, we keep track of the visitor’s history – each piece of content that they read.  When making personalized recommendations, we take into account all of this visitor’s personal history and retrieve content that is similar to all of their previously-read content.  We are careful to not recommend content that the visitor has already read.  For details on how to instrument personalization, check out the docs here.

Recommendations and Personalization Endpoints

GET /profile

Train a user profile for the purpose of personalized recommendations. Use HTTP method:related to return recommendations using the same UUID as specified here.

Query Parameters

  
uuidUUID for user profile being trained.
urlCanonical URL for page being trained against profile.
apikeyPublisher Site ID (API key)

Your API secret is not used with this endpoint.

Note: Profile data is kept for 7 days by default. Contact value@parsely.com for information about extending retention limits.

Retrieve a list of Post recommendations for a specified profile UUID (trained in HTTP method:profile) or a specified canonical URL.

If no sufficiently relevant recommendations are found, results are returned sorted by publication date. You can change this behavior by specifying the respect_empty_results parameter. Note that these results will still be limited by any filters specified, so it’s possible to have no results returned if there are none that match the filters, regardless of the value of respect_empty_results.

This endpoint takes one notable optional parameter:

  • sort, which determines the posts returned. _score orders by relevancy, and pub_date orders by publish date.

This allows you to find relevant recommendations but organize results to promote, for example, popular social stories.

Query Parameters

  
urlCanonical URL to use as the basis for contextual recommendations. Cannot be specified with uuid.
uuidUser profile ID to use as the basis for personalized recommendations. Cannot be specified with url.
apikeyPublisher Site ID (API key)

Your API secret is not used with this endpoint.

Optional Parameters

  
pub_date_startPublication filter start date; see Date/Time Handling for formatting details. No restriction by default.
pub_date_endPublication filter end date; see Date/Time Handling for formatting details. Defaults to current date and time if not specified.
limitNumber of records to retrieve; defaults to “10”.
pagePage number to retrieve if multiple pages are available; defaults to 1. Retrieving a page that is unavailable returns an empty record list.
sectionReturn recommendations that belong only in the specified section
authorReturn recommendations that only belong to the specified author
tagReturn recommendations that only belong to the specified tag
sortWhat to sort the results by. There are currently 2 valid options: _score, which sorts articles by overall relevance, and pub_date, which sorts results by their publication date. The default is _score
excludeExclude recommendations from a certain author, section or tag. The syntax is exclude=<meta>:<meta-value>, where meta is one of authors, section, or tags. To exclude all recommendations by author “Bob Yu”, the syntax would looke like this: exclude=authors:"Bob Yu". You can pass this parameter more than once. For example, to exclude all articles by author “David Austin” tagged “football”, you would pass exclude=authors:"David Austin"&exclude=tags:"football" in the request.
respect_empty_resultsBy default, if no recommendations are found, the API returns the most recent articles matching the filters. Passing this parameter with a value of 1 will return an empty result if no recommendations are found.
callbackJSON-P callback, a JavaScript function name that will be used to wrap the JSON response. The API also supports Cross-Origin Response Sharing.

The maximum pagination limit is 2000 records. You will receive an error from the API if you attempt to retrieve past record 2000 through any combination of page and limit.

GET /history

Retrieve a list of URLs visited by a user by UUID. This is the training profile being used for personalized recommendations.

Query Parameters

  
uuidUser profile ID to look up history on.
apikeyPublisher Site ID (API key)

Your API secret is not used with this endpoint.

GET /similar

In Beta

Just like the /related endpoint, /similar retrieves a list of Post recommendations for a specified profile UUID (trained in /profile) or a specified canonical URL.  Recommendations are based on the ranking algorithm.  For an in-depth description of the difference between the old and new ranking algorithms, read this blog post.

This method requires the same parameters as the /related endpoint and supports all the same optional parameters.

Examples

Contextual

Contextual Recommendations use a URL as input and return a list of relevant Posts as output.

The endpoint for this is /related and the parameter required is url. Let’s say the URL passed is about the Apple iPod. The returned “related” Posts might look like this:

{
  "data": [
    {
      "title": "Ipods And Crime"
      "url": "http://apikey.com/219045/",
      "author": "Kevin Black",
      "section": "Technology",
      "pub_date": "2012-02-20T19:08:00Z",
      "tags": ["apple", "ipod", "crime"]
    },
    // other similar JSON documents...
  ]
}

Personalized

Personalized Recommendations return Posts relevant to the user’s interests, and does not include Posts that the user has already read in the past.

Integrating Personalized Recommendations is slightly more complicated than the above APIs because it requires two steps on behalf of the implementer.

  1. Training. This logs that an individual with a specific UUID has visited a specific URL on your site.
  2. Recommending. Using a saved UUID for the user, return relevant stories from the Parse.ly API.

To train a user profile for your site, use the /profile endpoint. Pass the uuid and url parameters to log that an individual with that UUID has visited the specified URL.

Generating UUIDs is sometimes a point of confusion for our customers. Parse.ly already generates a UUID for every user using a cookie-based approach. This is accessible on pages with the Parse.ly tracker installed in the field PARSELY.config.parsely_site_uuid. You are free to re-use this UUID for your application if the cookie-based approach is sufficient for you.

However, some customers may have better UUIDs available. If you have e.g. an internal integer ID which also identifies registered/premium users on your site as well as anonymous visitors, this may be a better choice. Parse.ly does not care what the UUID is — it is simply a string used for identification on further calls to the API.

Once a user is trained, you can view that user’s history with /history endpoint. Pass the uuid and you will see the URLs currently associated with that user. This is mainly useful for debugging purposes.

Finally, call the /related endpoint with the uuid parameter to get personalized recommendations for that user. The default strategy uses a document similarity metric weighted toward most recently seen Posts, and a filter excluding all visited Posts; we plan to introduce additional strategies over time.

Personalized recommendations will appear as multiple posts, similarly to Contextual recommendations. For example, if you train a user on a story about 4G data access plans for mobile phones, you may end up with recommendations like this:

{
 "data": [
   {
     "title": "Verizon announces new 4G phone"
     "url": "http://apikey.com/244232/",
     "author": "John Peters",
     "section": "Technology",
     "pub_date": "2012-03-30T08:26:00Z",
     "tags": ["verizon", "technology", "phones", "4g"]
   },
   // other similar JSON documents...
 ]
}

Last updated: January 03, 2023

Last updated: January 03, 2023