Testing demand before implementation

September 15, 2020 by Adrian Durow . CRO

It is sometimes difficult to get commitment to implementing an idea.  Maybe there’s some doubt from certain stakeholders, even after you’ve presented evidence for an idea.  Maybe there’s reluctance to invest budget & time in the idea, particularly if it’s going to require a decent chunk of both.

Sometimes, user testing on 6 participants doesn’t provide enough confidence that an idea will work.

However… sometimes, we can implement ideas without actually fully implementing them – just to get a feel for demand for it.

 

Example: Live Chat

We’ve tested demand for Live Chat recently for a client.  3 of 6 participants in a usability testing session, mentioned that Live Chat would be a preferred method of contact.  But this was just 3.  From a sample size of just 6.  Even if this was 10 from 20, it would still be too small a sample to be fully confident in implementation.  Especially as Live Chat requires investment in a) a subscription and b) investment in chat handlers (or training for existing support teams).

So, we validated it first.  To assess demand from the 1000s of daily users to the site…

 

We used an A/B testing tool to code-in a floating Live Chat CTA like the one opposite.

Once clicked on, a modal was displayed: “Sorry, our Live Chat operators are not available at the moment.  Please either call (with number displayed), or complete an enquiry form (with enquire CTA displayed)”

We tagged these clicks as events in Google Analytics, and measured:

#1  Overall click demand

#2 In-context demand – i.e. which pages prompted these Live Chat clicks?  Which journeys prompted them?

We then were able to forecast conversion uplifts from implementing Live Chat, and were able to justify the investment properly.

Latest articles