What data does Parse.ly filter out?

Bot traffic

Parse.ly automatically removes all bot and spider traffic using the IAB/ABC International Spiders and Bots List, which identifies bots by user agent.

To deactivate bot filtering or add stricter bot exclusion, contact Parse.ly Support. Deactivating bot filtering will impact bot traffic in both Data Pipeline and Parse.ly Analytics.

Behavioral rate limiting

The behavioral rate limiting feature, if enabled, excludes visitor traffic that exceeds a certain threshold within a specified window. Parse.ly Dashboard excludes all events from any unique visitor site ID that views a single URL more than the specified number of times in a five-minute period. Excluded events include pageviews, videostarts, conversions, vheartbeats, and heartbeats. The default threshold, when enabled, is 600 events in 5 minutes.

Behavioral rate limiting is already applied to the Data Pipeline data.

To find out whether it’s enabled, or to enable/disable it, contact Parse.ly Support.

Top level domains and IP addresses

By default, Parse.ly only accepts traffic from a TLD that matches the Parse.ly Site ID (i.e., parsely.com). However, customers can either allowlist or blocklist other TLDs.

Parse.ly also maintains a list of IP addresses to ignore for every account individually, such as internal traffic.

Both Data Pipeline and Parse.ly Analytics ignore events from blocklisted TLDs or IPs.

Conversions

The Parse.ly Dashboard excludes any conversion from a visitor without a prior page view in the preceding 30 days because it is considered non-attributable and invalid.

This does not apply to Data Pipeline data.

Incorrectly formatted event data

Parse.ly also removes incorrectly formatted event data such as:

• events with null event_id, action, url, or apikey fields
• heartbeat events with a null engaged_time_inc field
• events with url fields that cannot be parsed as strings
• video events missing the video-metadata required “link” field