Steps for setting up an A/B test

Set Audience

Set Audience for your project and make sure that it is only true where it is supposed to be (i.e. where your code changes should execute). To do this use Preview and navigate the website.

NOTE! If you are working with / * no_doc_ready */ make sure that your classes/IDs of the elements you are changing are in the audience fact for page content.


Code the variation(s)

Code the changes according to project information and make sure that they work for all devices at all time (i.e. reloading the page, triggering the change multiple times, etc)


Set up Goals

Set up the KPI:s according to the project information and make sure that they work for all devices and both the original and the variation(s). Make sure that you get the Goal satisfied popup when performing the Goal task.


Quality Assurance

Before you send a project to review you need to go through the following steps to check that your project works: 


  • Ocular inspection of the variation = do tasks that are connected to the actual change and its location. Ex click/swipe/open/close element, add product to cart and go to checkout etc.


Goals and audience

  • Check the audience settings and that goals are triggered as they are supposed to. Both In the original and the variation on all devices that apply.


Specific pages

  • If changes are applied to a product listing and the test has a global audience. Review all different product listings on site, ex search result page, start page etc.
  • If changes are applied to a product detail page. Review different types of products to make sure it works on products with multiple colors/variants, no colors/variants etc


Languages ​​and currencies

  • If a website has multiple languages ​​check that the changes work with different languages
    • Languages ​​based on other sites (eg / en-no, / en-hk, / es-no etc.)
    • Correct links (href)
    • Correct checkout
    • Switch between different languages / countries (IMPORTANT !! that you actually have changed country and not just languages ex)
  • Test all currencies
    • Take into account while coding if prices are set with (,) or (.)
    • Correct currency abbreviation


Login and discount code

  • On pages with the option to login - try to login and see that everything is working
    • Tests in header
    • Tests on products
    • etc
  • Test with discount code
    • Test with / without discount code filled in
    • Important when we toggle discount fields for example


Devices and browsers

  • Test in all standard browsers on desktop (Chrome, Edge, Firefox, Safari)
  • Test if changes are responsive ( if the window size decreases / increases )
  • Test in Chrome's simulation of tablet and mobile in different sizes.
  • Test on your own mobile and in Browser stack on various devices, sizes and browsers. 
    • Apple: Chrome, Safari
    • Android: Chrome, Firefox, Samsung internet


Mockup / sketch

  • Double check sketch / test description if they match the changes that are done. If something does not work / can’t be done - tell a project leader
Was this article helpful?
0 out of 0 found this helpful