Title: DPL: Frequently Asked Questions
Author: staff
Published: August 1, 2022
Last modified: June 1, 2026

---

 1. [Data Pipeline](https://docs.parse.ly/data-pipeline/)
 2. DPL: Frequently Asked Questions

#  DPL: Frequently Asked Questions

## How to query all data from one day in a specific timezone?

Querying for a single day of traffic for a local timezone usually requires loading
data from multiple S3 buckets for the following reasons:

 1. Each S3 bucket name reflects UTC time
 2. Processing latency may delay the delivery of events so that they’re not received
    during the day they occurred

Typically, the events for a single day in a timezone should be confined to three
S3 buckets: the day that matches that timezone day, the day before, and the day 
after. However, in rare cases, events may be delivered up to 24 hours late. Parse.
ly recommends configuring ETL processes to continue importing these events and updating
tables and data warehouses accordingly. This can be accomplished by using `ts_action`
to represent event time instead of relying on the S3 bucket date.

Transforming data to reflect a specific timezone is dependent on the platform being
used. Amazon Athena utilizes the [AT TIME ZONE syntax](https://docs.aws.amazon.com/redshift/latest/dg/r_AT_TIME_ZONE.html),
and Google BigQuery has a [`timezone` input parameter](https://cloud.google.com/bigquery/docs/reference/standard-sql/functions-and-operators#datetime)
to transform dates and timestamps.

## How to replicate a calculation from the Parse.ly Dashboard?

The Data Pipeline provides a raw data feed of the same events that are used to power
the Parse.ly Dashboard. However, the dashboard calculations contain additional logic
that must be replicated in Data Pipeline queries in order to generate the same results.
Here are some factors to consider when attempting to reconcile page views, engaged
time, and video starts:

 * All data in the Parse.ly Dashboard is specified according to the local timezone
   configured in the dashboard preferences, while Data Pipeline data is always in
   UTC. The Data Pipeline data for a single day in a timezone will usually be found
   in multiple S3 buckets.
 * Apple News data is excluded from the Data Pipeline by default. To include Apple
   News data in Data Pipeline, contact [Parse.ly Support](https://docs.parse.ly/data-pipeline/support@parsely.com?subject=Add%20Apple%20News%20to%20Data%20Pipeline?).

## How to understand the expected latency for the Data Pipeline Standard loading events into S3?

The standard Parse.ly Data Pipeline inputs events into S3 buckets in [15-minute chunks with a maximum of 128 MiB per chunk](https://docs.parse.ly/data-pipeline-access/).
On rare occasions, latency between 1 and 24 hours may occur.

Latency is a system-wide [Parse.ly](https://docs.parse.ly/?output_format=md) metric
and is not specific to one customer. If Parse.ly is experiencing system-wide latency
with one of the tracking systems, then the Data Pipeline will also experience latency.
Possible causes could include a massive traffic spike or server issues. For any 
real-time latency delays, which directly impact Data Pipeline latency, see the [status.parsely.com](http://status.parsely.com)
page.

## How to explain missing session data?

Session data is not available for all distributed channels. For detailed descriptions
of the limitations of each channel, see the [Channel FAQs](https://docs.parse.ly/distributed-channels/).
A summary of this information is below:

 * The following channels **do not** support sessions. Find these in the [`channel` field](https://docs.parse.ly/data-pipeline-schema/):
    - Accelerated Mobile Pages (`channel = 'amp'`)
    - Apple News Real Time (`channel = 'apln-rta'`)
 * The Website channel **does** support sessions if the user or browser has not 
   blocked third party cookies.
 * The following integrations may or may not include session data depending on how
   they were configured by the customer:
    - [Mobile App tracking](https://docs.parse.ly/mobile-sdk/)

## How to interpret the value in the engaged_time_inc column?

Engaged time is defined and handled differently for pages and videos, with details
described in the [engaged time documentation](https://docs.parse.ly/engaged-time/).
In general, the `engaged_time_inc` field indicates how many seconds a user was active
either:

 * on the page, if `action = 'heartbeat'`
 * watching a video, if `action = 'vheartbeat'`

To correctly sum or aggregate total engaged time, use the fields `pageview_id`, `
videostart_id`, and `pageload_id` detailed in the [ID fields documentation.](https://docs.parse.ly/data-pipeline-schema/)

## How to explain why the visitor_site_id is empty?

Visitor data can be missing for any of the following reasons:

 * The following channel **does not** support visitors. This traffic can be identified
   in the `channel` field or the `url_domain` fields:
    - Apple News Real Time (`channel = 'apln-rta'`)
 * When a user or browser blocks cookies, that causes the `visitor_site_id` to be
   null.

For detailed descriptions of the limitations of each channel, see the [Channel FAQs](https://docs.parse.ly/distributed-channels/).

## How to define a Parse.ly session?

The Parsely JavaScript tracker uses a 30-minute expiry on the session cookie and
imposes no maximum length of time for a session. The session cookie is stored on
the host page’s first-party domain. This means that a device that visits pages on
the same parsely-integrated domain repeatedly with less than 30 minutes between 
each visit will create a session that lasts indefinitely. If a user returns to a
browser that has been inactive for more than 30 minutes, that will count as a new
session.

Note that sessions for server-side and mobile SDK integrations are up to the integrator
as those integrations are not created or maintained by Parse.ly.

## How to define what data is excluded from the Data Pipeline?

Parse.ly automatically removes bot and spider traffic, customer-specified IP addresses
to ignore, customer-specified domains to ignore, and incorrectly formatted event
data. For more details, see the [filtering documentation](https://docs.parse.ly/what-data-is-filtered-or-ignored-from-parse-ly-analytics-and-data-pipeline/).

## How to find the Parse.ly Data Pipeline data retention & archiving policy?

Parse.ly retains data for 3 months inside the Data Pipeline AWS S3 bucket and separately
for 13 months in the Parse.ly Dashboard. This is all specified in the contract.

## Why is some of the geolocation data missing?

Per Parse.ly’s geolocation provider, MaxMind, “Not all IP addresses can be geolocated
with enough specificity to return information about the subdivision, city, or metro
code of an IP address.” ([Source](https://support.maxmind.com/hc/en-us/articles/4414762983195-Country-level-and-City-level-Geolocation))

Test some IP addresses [here](https://www.maxmind.com/en/geoip2-precision-demo).

Last updated: June 01, 2026