Message Reports

Message Reports evaluate the performance of individual messages sent via the Message, Automation, or A/B Test composers.

See Reports for in-depth analysis of your use of Engage.

Navigation

Access message reports via the Messages menu.

Navigation varies per composer type:

  • Message: Messages Overview » History
  • Automation: Messages Overview » Automated
  • A/B Test: A/B Tests » History
Message and Automation
The view defaults to the History tab, which lists messages that you have already sent. Please note that this only includes messages that were delivered immediately. See the Automated tab for reports on Automated messages.
A/B Tests
The view defaults to the History tab, which lists tests that you have already sent.

Actions

Click an action icon at the end of a message row.

Eye = Preview Report
Expands the row, displaying metrics from the Message Report. Click the icon again to collapse the preview. Available metrics vary per message type. Click the question mark icon next to each for its definition.

Click Expand All and Collapse All to expose and hide Preview Reports for all listed messages.
Histogram = View Full Report
Opens the full Message Report or A/B Test Report.

Click your browser’s Back button or the view all messages link above the report title to return to the Message History list, Click your browser’s Back button to return to the A/B Test History list.

See Messages: Messages Overview and Messages: A/B Tests for additional information about these menu sections.

Metrics can take 10 - 15 minutes to update after our servers have received the event.

Report Detail

The report header displays the messages’s creation date, time, and time zone, and a panel of options.

  • View all messages: Navigation varies per composer type:

    Message: Returns you to Messages Overview » History.
    Automation: Returns you to Messages Overview » Automated.
    A/B Test: Returns you to the opened A/B Test Report.

  • Jump to: All options are listed, but the report will only display those related to the message type sent:

    Overview
    Notification Detail
    App Engagement
    Web Engagement
    Event Tracking

  • Download CSV: Click to download a CSV version of the report.

Message reports for Automated messages have the same data as shown here but appear in a different layout. Also see Automated Message Reports for information about their options panel.

Overview

The metrics displayed vary per message type and included platforms. Click the question mark icon next to each for its definition. The bar graph is a percentage breakdown of total sends per platform.

In-App Messages

If you send an In-App Message, the Overview metrics will include In-App Message Sends. These metrics represent the number of in-app and silent sends associated with the message. Silent sends are via Background Processing.

An In-App Message Send is logged for these scenarios:

  • An in-app message, with or without an associated push, is sent to an opted-out device. Because opted-out devices will only receive the in-app message, this is counted as an In-App Send.
  • An in-app message only is sent to an opted-in device.

An in-app message combined with a push notification that is sent to an opted-in device will always be counted as a Notification Send rather than an In-App Send.

Notification Detail

Report fields are dependent on the type of message sent.

  • Notification Text: The notification’s content.
  • Notification Action: The selected message action, along with associated tags.
  • In-App Message: The message’s content. This section is only included for in-app message only sends, or in-app messages combined with push notifications where the push notification text differs from the in-app message text.
  • Audience: The devices that received your message, along with associated segments and tags.

    Automated messages have an Automation heading, rather than Audience, with a summary of the message’s Trigger information, Status, and Creation and Last Modified dates and times. If the message status is changed from within the report, you must refresh the page to see the new Status under the Automation heading.

Additional headings and information will appear depending on the message’s original configuration.

In-App Messages

If an In-App Message was sent or combined with a push notification, the Total Impressions count appears below the Notification Detail, also broken down by opt-in status.

  • Total Impressions: The number of times an in-app message is displayed.

An In-App Message Impression is logged when a user encounters an in-app message. Either of the two scenarios that result in an In-App Message Send being logged will most likely also result in an Impression on open, assuming a user opens the app before the in-app message expires.

An in-app message combined with a push notification that is sent to an opted-in user can also result in an Impression; if the user opens the app without tapping the push notification, the in-app message will still display, counting as an Impression.

App Engagement

App Engagement appears in a message report when a push notification to a mobile app was included in the sent message.

The line graph is a comparison of Influenced Opens and Direct Opens over time. Hover over a point on the graph to display the counts. Total counts are below the line graph.

See Reports: Push Response Terminology for definitions of Direct and Influenced Opens.

Web Engagement

Web Engagement appears in a message report when push notification to a web browser was included in the sent message.

The line graph is a comparison of Web Sends, Web Clicks, and Web Sessions over time. Hover over a point on the graph to display the counts. Total counts are below the line graph.

  • Total Web Sends: The total number of web notifications sent to all devices.
  • Web Clicks: The total number of clicks on the web notification.
  • Web Sessions: The total number of sessions attributed to the push notification.

Event Tracking

If you have Custom Events enabled, the Event Tracking pane displays a list of events attributed, both directly and indirectly to the message.

  • Event Name: Human-readable name assigned to a particular Custom Event.
  • Notification Attribution: Displays whether your event was directly or indirectly attributed to the push notification.
  • Location: The source from which the event originates, most commonly one of Interactive Notification, Message Center, Landing Page, or Custom.
  • Count: Number of instances of this event.
  • % of Total Count: Count divided by the Total Count (listed on the bottom row).
  • Value: The value (monetary or otherwise) generated by the event.
  • Avg. Value: Value divided by Count.

Click the Download CSV button in the Event Tracking pane to export the report

Automated Message Reports

An Automated message’s message report has a small panel in the upper right corner, with options to manage the message and to filter the report view.

Automation Limits

Edit your App-level automation rule limits, which control over-notification to users who repeatedly meet automation criteria, capping automated messages per device per given time interval, e.g., a maximum of 4 messages every 10 hours.

Click the pencil icon to open your project’s Configuration settings. Click the Automation Limits tab, and make and save your desired changes. See Settings: Configuration: Automation Limits for detail.

Pause/Resume, Cancel, Edit, Duplicate

  • Active messages have the option to Pause. Paused messages have the option to Resume.
  • Cancel deletes the messsage.
  • Edit opens the message for editing.
  • Duplicate creates a copy of the message and opens it for editing. You must complete the composer steps in order to send the new message.

Filters

Make your selections from the date range and frequency dropdown menus. The report updates after each change.

A/B Test Reports

An A/B Test Report displays the test name and its Sent date, time, and time zone, as well as its Total Audience count. Variants are listed in the top table, with the Control Group below. Click a Variant name to view its individual message report.


Click the question mark next to a column header and its definition will display.


Data Export

To export engagement data for the entire A/B test, click the Download CSV button in the upper right corner.

Engagement data is sent to Urban Airship as soon as it becomes available. Often data is delayed because of connectivity issues with a user’s carrier, wifi, power, etc. You should wait at least 12 to 24 hours before acting on this data, to allow for potential lags.

The following data are included in the A/B Test CSV export:

Field Definition
Project Name Your project name
App Key Unique authentication key for your app
Push identifier Unique identifier that refers to the entire push send. The push identifier will be the same for each variant within an A/B Test, with N/A for users in the control group who do not receive a notification.
Sent Time UTC timestamp
Test Name User-friendly name for your A/B Test
Variant See: variant
Audience Description of the selected audience for the A/B Test. One of All Devices, Target Specific Users, or Test Devices.
Notification Alert Alert text for the given variant
Notifications Sent/Control Sample Size Audience send counts per variant/control group size
Indirect Opens Opens that occur within 12 hours of the notification, regardless of attribution
Direct Opens Opens that are attributed directly to interacting with the notification
Influenced Opens See: Reports: Influenced Opens


Event Tracking by Variant

If you are using Custom Events, the Event Tracking by Variant report tracks event conversions and associated values, broken out by variant or control group.

Click the Download CSV button in the Event Tracking by Variant pane.

The following data are included in the Event Tracking by Variant CSV export:

App Name App Key
Push Identifier
Test Name
Variant
Event Name
Notification Attribution
Location Count Value