Analytics: Main Concepts

First, let’s assume a conversation, we will base all further discussion on this following example where:

• “C” is the customer
• “B” is your Alcmeon bot
• “H” is a human, your advisor

It is from this example that we are going to define the vocabulary that you will find across our Analytics tool.

Other tools sometimes give very different meanings to the same words, that's why we think it is essential to give you the right definitions for using Alcmeon.

"Conversation" is a very important concept, which is used throughout Alcmeon. It has implications in Analytics of course, but also in other parts of Alcmeon such as Triage rules, Bots behavior and Satisfaction Surveys configuration.

To define a conversation, look at the delay between each Client message in the illustration above.

The first gap of more than 4 days (therefore 96 hours) occurs between Thursday 3rd at 7:30pm and Sunday 8th at 12:40am. Because of this gap, Alcmeon automatically closes the Conversation and starts a new one.

Please note that we are only talking about gaps between client messages. This means that we do not take into account your responses in this calculation. Your responses are part of the conversation but are never included in the calculation of gaps of more than 4 days.

Please also note that a Conversation gets a “closed” status only automatically, based on the rule defined above.

Conversation = All the messages from a customer on a social account, all spaced between them by less than 96 hours, whether they are public or private, and accompanied by any responses from the brand to these messages. Note: a conversation may not contain a response!

A customer sends you a message on a public channel, for example a comment on your Facebook wall. If you choose not to reply to this comment and the customer does not send another message for 4 days, then the single message, without reply, is considered as a Conversation. This is a very frequent case: think of users taking part to your contests for example.

"Answer" is a simple notion: it's a content sent by your brand on a platform like Facebook, Messenger, Twitter, etc.

You can see on the illustration above the Answers on a green background. So there are 5 in this example. Two of them were sent by a Bot and the next 3 by one or more human advisors.

"Answer" is a simple notion: it's a content sent by your brand on a platform like Facebook, Messenger, Twitter, etc.You can see on the illustration above the Answers on a green background. So there are 5 in this example. Two of them were sent by a Bot and the next 3 by one or more human advisors.

If the first message of the conversation is sent by a Bot before any message from the Customer, this first message is not considered as an Answer. This is the case on WhatsApp, Messenger, Instagram Direct and Twitter Direct Messages where your Bots automatically display a greeting and/or a main menu as soon as the chat window opens, without any intervention from the client.

Incoming flow is the set of messages that your customers have sent you.

It is therefore a fairly simple concept, but in the set of these messages, a large number of them are

• “automatic" messages linked to the use of your chatbots by your customers (indeed, a click on a bot button is actually considered as a message by messaging platforms)
• or messages that have been automatically archived by one of your Triage rules.

In the illustration above, the Incoming Flow will have 7 messages in it.

To monitor the incoming flow actually received by advisors, i.e. visible in the Inbox, you need to follow the additional metric:

‍

Incoming flow without chatbots

In the illustration above, the Incoming flow excluding chatbots will count 4 messages because :

• Message "1" was answered by a bot, so never visible in an advisor's inbox
• The message "Hello" was automatically archived when the next message from the customer arrived
• The message "I can't register" was automatically archived when the next message from the customer arrived

‍

Incoming flow = All messages sent by your customers, whatever the channel, whether they have been processed or not

Incoming flow excluding chatbots = All messages sent by your customers, whatever the channel, whether they have been processed or not. Excluded are messages that are automated exchanges with a bot (e.g. clicks on buttons) as well as messages automatically processed by a bot (e.g. archived or answered by a triage rule).

Indicators in this analytics page

• Messages received
  • This is the number of messages received during the period.
• Distinct clients
  • The number of clients who sent the messages received.
• Conversations started
  • The number of “Messages received” that are first messages of a conversation.
    • In the “Incoming flow” page, we are taking the number of "Messages received”, keeping only those that are first messages of a conversation.
      • Therefore in this page, the “conversations started” represents the number of conversations that started during the period.
    • In the “Incoming flow without chatbots” page, we are taking the number of "Messages received”, keeping only those that are first messages of a conversation.
      • Consequence: in a configuration with a chatbot scenario, that number is very small because in “Messages received” we are excluding those that were processed by a bot and never reached the inbox and most of these first messages were answered by a chatbot. So when we keep only first messages of conversation from that number that is already very low, the resulting number of “conversations started” stays low.
      • Therefore, in this page (compared to “Incoming flow”), the “conversations started” is NOT the number of conversations that started in the period in which there was at least one message received by an agent…
• Conversations received
  • The number of conversations of which at least one message is a “Message received” during the period. So they are conversations that may have started before the period but having one or more messages received during the period. What is done here is grouping the number of “message received” into conversations. The number of “Conversations received” is therefore always greater than the number of “Conversations started”.

Date of receipt

The period is called “Date of receipt”: therefore we are taking into account in this page the date when the messages have been received in Alcmeon.

One of the consequences of this is that it is not possible to compare the figures of Incoming Flow with the figures of Messages processed because the Incoming Flow is calculated over a period called "Date of receipt", and the Messages processed are calculated over a period called "Date of the action". However, a message can be processed one or more days after its date of receipt.

In the illustration above, the Incoming flow excluding chatbots will count 4 messages because :

• Message "1" was answered by a bot, so never visible in an advisor's inbox
• The message "Hello" was automatically archived when the next message from the customer arrived
• The message "I can't register" was automatically archived when the next message from the customer arrived

What does "processed" mean ?

In Alcmeon, processing a message means "performing a processing action". This does not mean that the processing of the message is "finished": it may not yet be "answered" or "archived".

The Alcmeon actions considered in this statistic as processing actions are: reply, archive, put on hold, annotate, request help, respond to a request for help, stop processing (ie. release a message), assign to another advisor, tag (qualification), route to another team.

Definition

Messages processed = All messages sent by your customers and which have been processed by a bot or by an advisor

Messages processed excluding chatbots = All messages sent by your customers and which have been processed by an advisor

Indicators in this analytics page

• Total Messages Processed
  • This is the number of messages having undergone one or more processing actions, and this action took place over the period. These messages may have been received on a date other than the date of the action.
• Answered
  • This is the number of messages answered, and whose response action took place over the period. These messages may have been received on a date other than the date of the action.
• Other columns
  • Metrics are the same as above but based on the action described, such as "Annotated" or "Archived"

Date of action

The period is called "Date of the action": we therefore take into account in this page the date on which the action was performed on a message.

One of the consequences of this is that it is not possible to compare the figures of Incoming Flow with the figures of Messages processed because the Incoming Flow is calculated over a period called "Date of receipt", and the Messages processed are calculated over a period called "Date of the action". However, a message can be processed one or more days after its date of receipt.

Note about the Total

As in all Alcmeon statistics pages, the totals are de-duplicated.

In particular for the Messages processed, the "Total messages processed" counts a number of messages: it is not the sum of the figures found in the following columns. Indeed, the same message counted as 1 in "Assigned", and as 1 in "Answered", will be counted as 1 and not as 1+1=2 in the Total.

This also applies to the Total of each column at the bottom of the table: the same message can be processed over several days, the figure indicated in total is not the sum of each line but the total number of messages.

First Human Answer delay is a measure of how quickly advisors respond to customer messages, from the customer's perspective. This period therefore does not take into account the non-working hours and days of your organisation. It is obviously computed only for messages answered by humans.

First human answer delay in Business hours measures this same time, but taking into account your business hours: the part of the time that is in non-business hours is subtracted from the calculated time. This allows you to measure time from your organisation's point of view, not from the customer's point of view. This is the indicator most followed internally by our customers, generally in the form of the percentage of messages answered in less than 2 hours (or less for the most ambitious).

First human answer delay in Business hours excluding routing refines the previous delay by allowing you to exclude from the delay the part during which the message was not yet routed to the team of the agent who answered (therefore the delay during which this message was invisible to him). This is a particularly interesting indicator in organisations where certain teams are only assigned messages when other teams route them manually: level 2 support teams for example.

Definition

First human answer delay = Time between when a customer sends a message, and when the first response from an agent is sent.

First human answer delay in Business hours = Time between the moment a customer sends a message and the time an advisor sends their first response, excluding the time in non-working hours from the calculation.

First human answer delay in Business hours excluding routing = Time between the time a message is routed to a team, and the time an advisor from this team sends their first response, excluding the time in non-working hours from the calculation and the time the message was not yet routed to the advisor’s team who answered.

What is the First Human Answer Delay per Conversation?

‍

This metric is used to measure the response time of a conversation: for each conversation it only concerns the delay with the first response of the conversation.

It is different from the delay of first answer described in the previous section and which concerns all replied messages of all conversations: here only the first replies of each conversation are taken into account, but in addition the starting point of the delay is the very first client’s message.

Indeed, since we are looking to measure the response time, it is necessary to measure from the customer's first message in the conversation, because it is possible that in the absence of a response, the customer becomes impatient and sends several messages successively. If the starting point of this delay was the message answered by the agent, this metric would not measure this period of waiting (and impatience) of the customer.

Definition

First human answer delay per conversation = Time between the customer's first message and the first human response in the conversation.

First human answer delay in Business hours = Time between the customer's first message and the first human response in the conversation, excluding the time in non-working hours from the calculation.

First human answer delay in Business hours excluding routing = the customer's first message and the first human response in the conversation, excluding the time in non-working hours from the calculation and the time the message was not yet routed to the advisor’s team who answered.

The Processing delay aims to measure the speed of advisers to "definitively" process a message, from the moment they attribute a message to themselves, by choosing one of the following actions:

• Answer
• Ask for help
• Answer to a help request
• Archive

We therefore always speak of "Processing time for the Response action", "Processing time for the Archiving action", etc. but never a processing delay only, because that wouldn't make sense in Alcmeon.

It is therefore a question of measuring, in general for the Response action, not only the speed of the advisers, but also the performance of the tools external to Alcmeon that they use, such as knowledge base tools, single customer reference , CRM, order or delivery management, etc.

Indeed, an "omniscient" adviser who would only have to type his response to the customer, would have a Processing delay for the Response action of a few seconds. This is rarely the case, since they generally have to look for the answer in other tools, and the delays observed are rather of the order of a few minutes on average.

This delay is calculated for each message having received one of the processing actions considered.

Definition

Processing delay = Delay between the moment an agent (or a bot) starts to process a message by clicking on the "Arrow" icon, and the moment he performs a processing action, such as "Reply" for example.

Processing delay per conversations = Sum of processing times for each of the messages answered in the conversation.

Processing delay per message with AI = Delay between the moment an advisor starts processing a message (by clicking on the arrow) and the moment he sends a reply having used an AI suggestion (either by using it again without modification, or by modifying it, or by not using it but requesting it anyway).

Processing Delay per message with AI

Analytics for processing delay per message with and without AI compare the average processing delay when an advisor does or doesn't use AI.ssistant to write an answer.

• Answer without AI.ssistant
• Answer with AI.ssistant (the advisor to ask for a suggestion)
  • Answer with original AI.ssistant suggestion (the advisor sends the AI suggestion unchanged)
  • Answer with modified AI.ssistant suggestion (the advisor modifies the AI suggestion before sending it)
  • Answer with discarded AI.ssistant suggestion (the advisor indicates that the suggestion does not satisfy him/her and writes his/her own response)

You can click on each delay to see the details of the messages used to build this average.

You can also export this information to compare the suggestion proposed by the AI and the response finally sent to the customer

A modification can be slight (adding an introductory or concluding sentence) or very significant (rewording the entire suggestion)

It is important to interpret these statistics in the light of the corresponding volumes in › Volumes › AI.ssistant messages

This is to measure the average time to "close" a conversation.

The customer's last message in the conversation is excluded from the calculation, to avoid artificially extending this closing time due to customers thanking the advisor several days later.

Definition

Conversation duration = Within a conversation, the delay between the first message from the customer and the last reply by a bot or an advisor. Only closed conversations are taken into account.

Processing delay per conversations = Within a conversation, the time between the first message from the customer and the last response by a bot or an agent, excluding the non-working hours part of this time from the calculation. Only closed conversations are taken into account.

FAQs

Why is the displayed total of a column different from the sum of its rows?

In the vast majority of Alcmeon statistics pages, the data table displays a column total which is different from the sum of the rows of this column.

Take this table of results for example:

‍

In the "Total conv. processed" column, we have the sum of the 2 lines for January and February: 41249 + 28238 = 69487 which is lower than the total displayed 67865.

41249 is the number of conversations with at least one message processed in January 2021.

28238 is the number of conversations with at least one message processed in February 2021.

Some conversations spread out over time, so they had message A processed in January and also message B processed in February.

One of these conversations therefore counts as 1 in January, and as 1 in February. However, it will count as 1 in the total of the column: we do not count it twice. Thus, you are assured that the total displayed is indeed a number of conversations.

Note 1: same for all column totals of this table

Note 2: ditto for the “Total conv. processed" in a row that is not equal to the sum of digits in the other columns in that row. Indeed, a conversation may have been treated with several of these actions. For example tag positioned + assigned + answered.

When I add the incoming flows XXX and I compare with the processed messages YYY, why don’t I find the same result?

This type of question generally boils down to a misunderstanding of the stats pages studied, and more particularly of the dates in play.

For example, in incoming flows we have “Date of receipt” as the main criterion, while in processed messages we have “Date of action”. It is therefore absolutely impossible to compare these figures with each other, and the differences are sometimes even gigantic! This is because messages are sometimes processed days after they are received.

Other pages are in this case, you must pay attention to the name of the date criterion at the top left.

When I go back X years, why do some analytics pages no longer count anything?

For the following analytics pages:

• Conversations with Chat Bots
• Custom Events

Indeed, if we look at the stats from several years ago, we no longer have any figures. This is due to the anonymisation of messages and users (due to RGPD regulations in particular). For eg; for an Alcmeon environment where the anonymisation period is defined at 2 years, then for these statistics pages which have unique users, therefore use the identifier of each user, then after 2 years these identifiers disappear and the stat becomes impossible to calculate.

There is no workaround possible to find this old data.

What is the difference between the criteria “Team” and “Team (current tag)”?

Team = the team the message was routed to when the action was performed. For a response action, it is therefore equivalent to “the team of the advisor who responded”

Team (current tag) = the team the message is currently routed to.

So, in the case where a message has been answered by a team, then routed to another team, the difference in the calculation of statistics (such as Volumes of messages processed, or Response times) is significant depending on whether you select either of these two criteria.

In 99% of cases we recommend using the “Team” criterion.

Conversations (monthly report): why is a message not counted at the day it was processed?

This analytics page is an exception among our analytics pages: a conversation is only counted once across all days of the selected month.

If I select the month of June, for a conversation with 2 messages replied on June 10 and June 13, the conversation will only be counted on June 10 and not on June 13.

‍