Experimentation as Opposed to Intuition, New paradigm in Business Decision Making

Companies are increasingly using experimentation to not just validate their decisions but to find new ways of working by giving a free rein to their employees to think of a new way of doing things and conduct experiments to validate their hypothesis. In these companies, it is no longer the prerogative of top management alone to think of improvements and conduct experiments to validate their hypothesis.

The culture of experimentation is very evident in companies using digital interfaces and platforms to engage customers, since the firsthand contact between customers and company personnel is absent and the only way to gauge customer response is through experimentation.

The companies which are leading this style of working are digital platform corporations Microsoft, Google, Facebook, Amazon, Bookings.com and Expedia.

When we perfectly reconstruct from an original file by reducing the file size without causing any loss of information is called as Lossless Compression. Essentially, lossless compression removes redundant information which is not required. Contrary to lossy compression, it doesn’t remove unwanted information from the file which is subjected to compression. The lossless compression algorithms basic principle is easy to grasp. Imagine a file made of 6 M’s in a row, which would look like this: “MMMMMM”. You could compress that to take up less space by replacing those six characters with something like “6M”. Formats such as MP3 and JPEG used lossless compression by default.

Generally, you should use lossy compression if you are okay with some information loss, which may not be exactly the same as the source file and lossless compression when you require a perfect reconstruction from the original file without any compromise on information loss. For Windows there is software called 7-Zip which is free and can be used for compressing different file formats and PeaZip is another free software which supports many different file formats for compression.

So strong is the culture of decision making by experimentation that senior executives are no longer expected to make decisions not supported by experimentation. In these companies thousands of decisions are made every day by people across the organization at different levels. The companies in the lead of this culture permit their employees to conduct research with very little authorization. They believe that the need to receive validation would itself curb the zeal to experiment. Expedia and Bookings.com conduct hundreds of concurrent experiments at any point of time. Even companies which are not strictly online platforms such as FedEx and H&M have embraced online testing.

One of the companies which has raised experimentation to the level of a cult is Bookings.com, the largest Online Travel Agency in the world. Bookings.com permits almost anybody to run an experiment on its portal. Experiments could range from changing the price offer to test price sensitivity to changing the colour of the background to test consumer response. It could be the language used to instill a sense of urgency (rush… last 3 rooms left!) or the size of the cross to be clicked to close a popup.

This form of testing is called A/B Testing, meaning that it tests response of one option ‘A’ over another, ‘B’. In this process the experimenter changes just one aspect of the website and makes a comparison with the previous version under standard conditions, when all other conditions remain unchanged. Most hotels would like an Online Travel Agency to perform the tests because the customer traffic on an OTA is much higher than the traffic on the hotel’s website. The much larger sample size makes the validity of the test results mush higher.

The features of the hotel that need to be tested and are likely to have the most impact on bookings are the tariff and images of the hotel. Expedia found that a room with a nice view makes a big difference. Great exterior pictures make the hotel attractive. Perks and extras like breakfast, conference facility and transport are features that are tested. When to pay, now or after seeing the property can be a great inducement to book. This is similar to the return policy of an online retailer. It inspires confidence, on the other hand permits last minute rejection and consequent loss of revenue.

In one instance of testing, Bookings.com listed the same property under two different tariffs to see which tariff received more bookings, in short, how price sensitive was the offering. The OTA could gain enormously from the knowledge derived from such experiments. They could guide hotels they wish to favour in order to achieve maximum bookings. Online retailers also experiment. Amazon found that while selling a game Air Patriots where the players defend a structure against air bourne attackers, the retention rate dropped by 70% and revenues fell 30% when the difficulty level of the game went up by 10%. Amazon responded to reverse the difficulty level which partially corrected to consumer response.

There are ethical considerations in conducting experiments.

Facebook ran a test involving 690,000 users to test whether emotional states were contagious. Did reading fewer positive news stories cause people to make fewer positive posts. Did fewer negative stories lead to fewer negative posts? The social media platform created an algorithm which posted positive and negative stories to a test group of 300,000 users and compared it with a control group of 300,000 users not subject to such tests. The test came under criticism because it amounted to manipulation of emotions of people.

LinkedIn has a policy of not conducting experiments that would deliver a negative user experience or have a goal of altering the member’s emotional state. All consumer tests studying human behavior are to some extent manipulative in nature. Experimenters need to know where they must draw the line.