Message Reports evaluate the performance of individual messages. The format of a message report varies depending on how the message was created. A/B Tests have an overview report along with a message report for each variant.
These navigation steps are for accessing message reports only. See the Messages Menu Guide for full menu documentation.
Open your project from the dashboard, then click Messages and select Messages Overview or A/B Tests.
Messages Overview contains message reports for messages that were created via the Message, Automation, and In-App Automation workflows. Message reports are accessible from three tabs: History, Automation, and In-App Automation.
After 14 days, messages with Expired or Stopped status move from In-App Automation to the History tab.
A/B Tests contains overview reports for messages that were created via the A/B Test workflow only. The overview reports are accessible from the History tab. A message report for each test variant is linked from within the overview report.
Open a Report
Click a report action icon at the end of a message row. In-App Automation and A/B Tests have an icon for the Full Report only.
- Eye = Preview Report
- Expands the row, displaying metrics from the message report.
Click the icon again to collapse the preview. Available metrics vary per
message type. Click the question mark icon next to each for its definition.
Click Expand All and Collapse All to expose and hide Preview Reports for all listed messages.
- Histogram = View Full Report
- Opens the full message report or A/B Test Report.
Metrics can take 10 - 15 minutes to update after our servers have received the event.
Reports for messages created via the Message, A/B Test, Automation, and In-App Automation workflows, and API counterparts contain the following components.
The report header contains a panel of options in the upper right corner and links to report sections at the bottom. Additional information displays slightly differently depending on which workflow was used to create the message. Those differences are described for each workflow type.
Message and A/B Test
The header displays the message's notification text and creation date, time, and time zone. The options panel is a single Duplicate button.
- Duplicate: Creates a copy of the message and prompts you to choose a workflow for the new message. You must complete the workflow steps in order to send the new message.
The header displays the message's notification text and creation date, time, and time zone, as well as the message status and Automation Limits status. The options panel is in the blue banner and has a date range filter and controls to change the status of, edit, or duplicate the message.
Edit your App-level automation rule limits, which control over-notification to users who repeatedly meet automation criteria, capping automated messages per device per given time interval, e.g., a maximum of 4 messages every 10 hours.
Click the pencil icon to open your project’s Configuration settings. Click the Automation Limits tab, and make and save your desired changes. See Settings: Configuration: Automation Limits for detail.
Pause/Resume: Active messages have the option to Pause. Paused messages have the option to Resume.
Cancel: Deletes the message.
Edit: Opens the message for editing.
Duplicate: Creates a copy of the message and opens it for editing. You must complete the workflow steps in order to send the new message.
Make your selection from the date range dropdown menu. The report updates immediately after selection.
The header displays the message's name and status. The options panel has controls to change the status of or duplicate the message.
Active messages have the option to Stop. Stopped messages have the option to Restart.
Stop: Marks the message as canceled. This is effectively the same as specifying an end date, immediately applying an end date of "now." Active messages can be canceled at any time.
Restart: Changes the status of a Canceled or Expired message to Active. See: Restart Message. This is effectively a shortcut to editing the message and either eliminating or specifying an end date. Expired messages can be restarted within a grace period of 14 days. After 14 days they can only be duplicated.
Edit: Opens the message for editing. Availability depends on message status. Active messages can be edited at any time. An Expired or Canceled message can be edited and its end date extended, making it active again, within a grace period of 14 days. After 14 days they can only be duplicated.
Duplicate: Creates a copy of the message and opens it for editing. Yo must complete the workflow steps in order to save the new message.
Click the arrow on the right side of the page to expose the message's API Payload.
The content included in the API Payload is what we call a push object. The push object describes everything about a push notification, including the audience and push payload. For more details on the push object, see: API: Push Object.
In-App Automation does not have a public API, so the API Payload is not available for those messages.
Latest Message Activity
Latest Message Activity shows the username, date, time, and activity associated with the most recent change to the message. Last Message Activity data is available for 90 days after message creation. Click Show all to open a modal window that displays all message activity. This is only an option for messages with more than one activity.
Directly below Latest Message Activity are statistics panes. Click the question marks to see term definitions appear as tooltips. Note that Direct Engagements will not appear if the message included a send to an Open Channel.
Engagement data is available for all messages sent via push. Click the Download CSV button to export the data. Reports created via Automation have a time range dropdown menu instead of the Download CSV button. The menu options are dependent on the lifetime of the message, and the report updates immediately after making a selection.
Engagement data does not apply to or appear for In-App Automation.
The bar chart includes engagement data for 12 hours following a message being sent. By default, the chart displays Direct Engagement for the message sent, e.g., a direct open on a mobile app or a click on a web notification.
Check the boxes to see additional information:
Channel Breakdown: Displays the chart data broken down by channel, e.g., iOS, Android, and Web.
Attributed Engagement: Displays the message's total engagement attributed to the notification.
The chart remembers your selected preferences. If you visit a message report and enable Channel Breakdown, the next report you view will default to this presentation as well.
Below the chart are metrics tables for Channels, as well as In-App Message and Message Center (Rich Page), if included in the message.
Channel: Notification sends and engagement metrics by channel.
In-App Messages: Total number of message views by opt-in and opt-out status.
Message Center (Rich Page): Total number of message sends, views, and deletions.
As with other terms on the page, click the question marks to see definitions appear as tooltips. Additional notable definitions:
- In-App Message Send
- The number of in-app and silent sends associated with the message. Silent sends are sent via Background Processing An optional delivery feature that wakes the app and gives it some running time to perform work, such as downloading content for future display. Data included in the push notification is available for background processing, e.g., to send URLs and then download that content. iOS only. .
An in-app message Send is logged for these scenarios:
- An in-app message, with or without an associated push, is sent to an opted-out device. Because opted-out devices will only receive the in-app message, this is counted as an in-app message Send.
- An in-app message only is sent to an opted-in device.
An in-app message combined with a push notification that is sent to an opted-in device will always be counted as a notification Send rather than an in-app message Send.
- In-App Message View
- The number of times an in-app message was displayed.
An in-app message View is logged when a user encounters an in-app message, whether the in-app message was sent by itself or in combination with a push notification. Either of the two scenarios that result in an in-app message Send being logged will most likely also result in a View on open, assuming a user opens the app before the in-app message expires.
An in-app message combined with a push notification that is sent to an opted-in user can also result in a View; if the user opens the app without tapping the push notification, the in-app message will still display, counting as a View.
- Attributed Web Sessions
- The total number of sessions attributed to the push notification.
An attributed session is a session that occurs within a 12-hour window of a web push notification. Sessions are generated when a user directly visits the website with the Urban Airship Web SDK present, or by clicking a web notification that leads the visitor to the site. The page the user visits must have the Web SDK present in order to track sessions.
If a user visits the web page within 30 minutes prior to when the web push notification arrives, and they:
Click the web push notification, a new attributed session will be generated.
DO NOT click the web push notification, an attributed session will be generated only if there has been 30 minutes of inactivity on the website.
Message Detail is the same as shown in the Review step of a workflow, so the content and display will vary depending on the workflow used to create the message.
Click the arrows to page through the various previews. The channel and display type dynamically update in the dropdown menu above. You can also select a preview directly from the dropdown menu.
Delivery by Time Zone
The Delivery by Time Zone table is included for messages delivered according to a user's local time zone.
Metrics for Alerting Sends and Engagement are provided for each time zone. Below the table is the the total number of time zones with zero sends; this represents the number of time zones where none of your message recipients are located.
Send a message to a user's local time zone:
For example, a push notification scheduled for 9 a.m. will arrive for people on the east coast at 9 a.m. Eastern Time, in the midwest an hour later at 9 a.m. Central Time, then on the west coast two hours after that, at 9 a.m. Pacific Time.
If you have Custom Events enabled, the Events pane displays a list of events attributed, both directly and indirectly to the message. Click Download CSV to export the data.
- Event Name: Human-readable name assigned to a particular Custom Event.
- Notification Attribution: Displays whether your event was directly or indirectly attributed to the message.
- Location: The source from which the event originates.
- Count: Number of instances of this event. % of Total Count: Count divided by the Total Count (listed on the bottom row).
- Value: The value (monetary or otherwise) generated by the event.
- Average Value: Value divided by Count.
A/B Test Report
A/B Tests have an overview report along with a message report for each variant.
To open an A/B Test report, open your project from the dashboard, then click Messages and select A/B Tests. On the History tab, click the histogram icon at the end of a message row. Also see Navigation above.
The header displays the test name along with the sent date, time, and time zone. A Download CSV button is in the blue banner. See: Data Export.
Performance contains statistical data for each variant and the control group. Click a variant name to view its individual message report.
Message Detail is the same as shown in the Review step of the A/B Test workflow. It is also the same as displayed in the Message Detail section of the individual message report.
Click the arrows to page through the various previews. The platform and display type dynamically update in the dropdown menu above. You can also select a preview or variant directly from the dropdown menus.
Click Download CSV and select Performance Data or Variant Event Data.
Engagement data is sent to Urban Airship as soon as it becomes available. Often data is delayed because of connectivity issues with a user's carrier, wifi, power, etc. You should wait at least 12 to 24 hours before acting on this data to allow for potential lags.
The following data are included in the A/B Test Performance Data export:
|Project Name||Your project name.|
|Key||Unique authentication key for your app.|
|Channel Identifier||Unique identifier that refers to the entire push send. The channel identifier will be the same for each variant within an A/B Test, with N/A for users in the control group who do not receive a notification.|
|Sent Time (UTC)||UTC timestamp.|
|Test Name||User-friendly name for your A/B Test.|
|Variant||The letter corresponding to the message variant. A, B, C, etc.|
|Audience||Description of the selected audience for the A/B Test. One of All Users, Target Specific Users, or Test Users.|
|Notification Alert||Alert text for the given variant.|
|Notifications Sent/Control Sample Size||Audience send counts per variant/control group size.|
|Indirect Opens||Opens that occur within 12 hours of the notification, regardless of attribution.|
|Direct Opens||Opens that are attributed directly to interacting with the notification.|
|Influenced Opens||See: Reports: Influenced Opens.|
Variant Event Data
If you are using Custom Events, you can download a report of event conversions and associated values, broken out by variant or control group.
The following data are included in the A/B Test Variant Event Data export: