Conversion Elite June 2018 – Recap and Key Takeaways

To get a return you have to invest and the Conversion Elite 2018 conference in London absolutely smashed it again with awesome speakers and a fantastic networking opportunity for the CRO and SEO community. Never underestimate the value of meeting up with people who live and breathe CRO and SEO on a daily basis.

For me the conference can be summed up with some universal truths that we should not forget:

  • That for many people experimenting does not come naturally. People are loss averse and don’t like failing.
  • To try and understand the customer’s back-story. What is the underlying motivation for their behaviour?
  • You can only personalise the customer experience if you have repeated interactions to collect data, frequent interactions to make personalisation relevant and the capability to detect the customer.
  • To revisit hypothesis for ‘failed’ experiments because it may be the execution that made it fail or the context within which it was implemented.
  • That good data that you and your organisation can trust makes money
  • To fix the fundamental issues with your website as otherwise you will just be burning money.
  • About your user’s emotions and whether your site results in a positive experience as otherwise customer won’t return.
  • To embed your CRO team within your agile business to collaborate with your IT developers rather than see them as a barrier to CRO.
  • The customer and that whatever they see is processed by their brain and will likely influence their behaviour whether they are consciously aware of it or not.

Guido Jansen – Psychologist, CRO Global Lead:

Guido talked about the challenges of creating a culture of experimentation within a client—side organisation. He pointed out that whilst A/B testing is all about making small incremental improvements our brains don’t work like that and we would prefer to make large improvements in one go. However, if every week we achieved a 0.5% improvement in the success metric that adds up to a 30% uplift over a whole year.

To be successful with CRO it is necessary to change the culture so that people are comfortable with the fact that most tests are likely to fail. Get people to trust the data and so they can trust the experiments.

Neither can you do everything in a centralised team. To create the right culture it is necessary to break down silos, show case results and build a community to involve other teams in the organisation.

“Build the rituals so that they are comfortable with what they are doing.”

Translate your tests into a return on investment as that’s what your business wants to see. As you develop the culture you can move away from pure A/B testing to validate features and functionality. However, if you are struggling to get traction with A/B testing in some regions or countries consider bringing in an agency to kick start experimentation and prove the value of testing.

Anna Dahlström – UX FIka:

Anna’s presentation focused on the importance of story-telling and how stories have the ability to motivate users and lower our intellectual guard. CRO is about connecting emotionally with users and stories facilitate this process.

It’s necessary to understand the context of behaviour to appreciate the full user story. This involves exploring the plot, characters and décor of the story. The plot or user story allows us to map out the emotional experience and user experience maps are a brilliant tool to understand how the user feels and break down silos within the organisation.

Good characters resonate and evoke emotion. If we understand our audience through developing personas and other techniques it is much easier to design for them. Increasingly we may look at characters in terms of players and actors, such as devices and the smart home (i.e. the Internet of things).

Décor is about sign posting or guiding users, but increasingly users need to find their own way. Indeed, website owners are no longer in control of the user experience as Google and social platforms are often the default homepage for the beginning of a journey. For e-commerce many searchers start on Amazon.

Stephen Pavlovich – CEO of Conversion .com:

Stephen broke with convention and explained that personalisation is not for everybody. It can be a competitive advantage for some companies, but it is important that organisations prioritise their focus and don’t look at solutions before defining the problem.

For companies to be ready for personalisation they need to meet at least one of these criteria.

  • Have repeated interactions to be able to collect data on their users,
  • Frequent interactions to provide the opportunity for personalisation,
  • Or a diverse audience to make segmentation feasible.

Companies that use personalisation also need to have a good understanding of their customers to implement it effectively. A history of A/B testing also helps because this allows you to begin analysing the behaviour of different customer segments. But companies must also have the capability to detect customers to serve a personalised experience.

The danger with implementing personalisation when a company is not ready is that it will create a fragmented customer experience. You can download a free white paper from which outlines a 4-step process for creating an effective personalisation strategy.

However, for those companies that are able to implement personalisation properly it can be a major competitive advantage. Stephen mentioned several brands that has helped to implement successful personalisation campaigns, including Domino’s Pizza, where they managed to increase the upsell of side orders by 20%.

Divya Isaiah – Channel 4:

Divya presented details of how Channel 4 has made literally thousands of user validated decisions by employing user research and A/B testing to change their website. They use all the tools in the toolbox to inform decision making and find that by asking stakeholders to put their ideas in the form a hypothesis they manage to prevent most weak suggestions being taken forward.

Channel 4 have also used customer experience maps to good effect as this allowed them to identify what users most care about. But also this identified where the gaps were by breaking the experience up into the component elements and allowed them to identify a single metric to measure.

Other insights shared by Divya included:

  • Know you devices as the experience and user goals are different,
  • Test your assumptions and rules,
  • Share learnings.

One of the achievements that Divya is most proud of is the online archive of all their test results. This includes a one page summary of each test which ensures their knowledge stack is up to date and relevant. This allows users to view what they have already done in that space to inform any future changes and tests.

Anna Lewis – Polka Dot Data:

Anna framed her presentation around don’t forget your data as good data makes you money. Remember data collection first and what the data tells you about your site. It’s essential to have a measurement framework in place as hypothesis work best when the KPIs are derived from the business goals.

You need to be able to trust your data so that your organisation can trust what it tells them. Always apply segments to your data because aggregated data is crap. Look at all your segments and apply to each step of the user journey.

Specify all your events so that you can plan and set up event tracking to monitor all engagements that matter. These should include:

  • Engagement – Clicks on relevant elements (e.g. hero banner),
  • Micro conversions – Products/add to cart
  • Macro conversions – Purchase/lead generation


In Google Analytics companies can also now use advanced ecommerce and shopping behaviour which enables you to apply segments to. Checkout behaviour also allows you to apply segments and for non-ecommerce sites you can still use this function for tracking lead generation activity.

Craig Sullivan – Optimise or Die:

Craig delivered an entertaining presentation by demonstrating how most websites can easily save thousands if not millions of pounds by spending 15 minutes to audit device flows. Don’t kid yourself that your website or app is working perfectly as all sites have product defects. However, nobody is likely to call you about it and your call centre won’t pass anything onto you unless it relates to their own KPIs.

The cost of these defects can be huge and yet the effort to detect them is very low. These problems are often basic and avoidable. If you don’t fix these issues then when a customer comes across them they will think you don’t care about them.

“If you don’t check you don’t know.”

Craig shared a number of free tools he and others have created to make your analysis more automated. These include a spreadsheet for your GA data and a sheet for predicting how much money you will have saved. His Profit Grid sheet also automatically creates dimensional funnels.

So, stop unintentional discrimination as this reduces the size of your audience. Accessibility is about getting maximum reach and it drives customer delight and makes you more money.

Neil McKay – Endless Gain:

Neil followed on from Craig’s presentation by demonstrating how the customer experience is the perception of how a brand is treating the user. He argued that emotions are the driving force behind what makes people buy. By using biometrics we can understand how sites make customers feel and better predict behaviour and outcomes.

Biometrics involves statistical analysis of biological data such as facial expressions, eye-tracking data, a user’s pulse and brain activity (EEG). These techniques can be used to track user’ s sense of anger, disgust, sadness and other emotions as they browse or try to complete a task.

By measuring emotions when conducting usability testing this allows the client to see how the customer feels and better understand the impact of the user experience on their behaviour? By using this approach Endless Gains can keep re-testing new design experiences until they detect an improvement in the user’s emotional state. This can then be validated with an A/B test to confirm if it translates into an uplift in the success metric.

Simon Elsworth –

Simon gave a great insight into how to market CRO to developers. When he started out at Sky everyone worked in silos and so they implemented changes in isolation, set different goals and used separate code. There was no opportunity to optimise and changes were slow, expensive and risky.

The plan was to bring all of digital together and create a centre of excellence in Leeds as part of a digital transformation programme. This allowed the CRO team to embed themselves into the agile development teams. They now work with product, participate in daily stand ups and go to sprint reviews. There was no need to promote agile working to the CRO team as they already worked in an agile way.

This allows the CRO team to inform developers how much extra money their changes have made for the company. This helps motivate the developers and the CRO team now also speak their language. It has been important to train the developers to ensure they use the tools in the right way and follow the correct procedures to maintain confidence in the testing process.

“Don’t get emotionally attached to your ideas as they will fail”

Sky find that genuine customer frustrations that are fed to them from the customer contact centre provides ideas for many of their hypothesis. This may have helped their 75% success rate for experiments.

Simon also is a great believer in asking their technology suppliers (e.g. Optimizely and Creative CX) for help. Get them to come into your office and pick their brains on how to get the most out of their tools.

Bart Schultz – Online Dialogue:

Bart is a pioneer of online psychology and delivered an insightful presentation on how the brain works and what this means for optimisation. Bart demonstrated dual processing (i.e. System 1 and System 2) and how more decisions happen outside of what we consciously control. System 1 is our quick, automated, way of making decisions which is largely outside of conscious awareness and System 2 is our slow, more rational decision making process.

Sometimes we can control our automated answers from System 1 by deliberative thought (i.e. System 2). However, System 2 easily gets depleted, particularly if we are tied up with mental maths. There are three important characteristics of System 1:

  • It’s always on – for evolutionary reasons it can’t be switched off. For example if a PPC ad is shaped in the form of an arrow and they keywords help form this pattern researchers have seen a 25% uplift in clicks.
  • It’s an associative network and so we can’t stop linking words or images with related concepts.
  • What You See Is All There Is (WYSIATI) means that a design can still positively influence a user even it consciously annoys them. When something is present it will be processed (e.g. a random number that acts as an anchor for a price).

System 2 is our conscious mind and is switched off at night. The three characteristics we need to consider for optimisation are:

  • It has limited capacity and so can be depleted to get people to give an automatic answer instead.
  • It requires attention. We can only focus on one thing at a time as we are unable to multi-task. This means that we can be distracted to force an automatic answer.
  • We are not aware of the effects on our decision making due to post decision rationalisation.

When we ask people questions, such as user research, we only get answers from System 2. This is why people will say one thing and then respond differently in reality.

Other insights Bart shared related to improving the effectiveness of optimisation through:

  • Velocity over value
  • Setting an overall evaluation criterion (OEC) to target rather the conversion rate.
  • Effectiveness over efficiency

Finally, Bart suggested measuring ethical metrics to try and understand if your tests are resulting in any socially adverse outcomes.

The conference ended with a panel discussion and Q&A session with some of the speakers expressing their views on industry issues. Free drinks provided by the sponsors ended the day on a good note!

About: Neal Cole is the founder of Conversion Uplift, a digital marketing consultancy which specialises in marketing strategy and conversion optimisation. Neal previously established A/B testing roadmaps and optimised sites for Shop Direct (e.g. and (e.g. and before going freelance in 2016.

100 thoughts on “Conversion Elite June 2018 – Recap and Key Takeaways

Leave Comment