Experimental Methods

Methods can be qualitative or quantitative.

Qualitative methods help you answer the questions "What?" "How?" and "Why?"

One drawback of qualitative methods is that they're subject to the cognitive bias of interviewers and respondents.

Additionally, the results obtained from qualitative research won't help you assess the impact of changes in numbers. Nor will it help you summarize results, as the sample will be too narrow.

Quantitative methods answer the question "How much?"

They help you determine the extent to which changes affect user behavior with considerable precision. However, they have certain limitations with respect to the interpretation of results and understanding users' motivation.

Qualitative Methods of Testing Hypotheses

Depth interviews

Depth interviews are characterized by a high level of trust between the respondent and interviewer. This allows the user's hidden motives or needs to be revealed.

Expert interviews

An expert interview is a type of depth interview with an expert in the field being studied.

Problem interviews

Problem interviews are a type of poll or depth interview.

UX analysis

This group of methods helps determine what problems users face when using the interface. Hypothesis testing is an essential part of any UX analysis.

Focus groups

A focus group is a group discussion where participants share their thoughts on a subject or product. Often, several homogeneous groups are formed, this is to help you understand what different users think.

Qualitative Methods of Testing Hypothesis

Polls and surveys

Polls can be conducted both in person and remotely: by phone, on polling websites, or via messenger or email. The main advantage of polls (and online polls in particular) is the high level of precision. Anonymous respondents are more likely to be honest.

Data analysis

Sometimes a hypothesis can be directly or indirectly tested using existing data. Existing data on user behavior can be useful in prioritizing hypotheses.

A/B testing

Traffic is split into two or more groups. Everyone is shown a particular version of the page, and data is collected on pageviews, clicks, and purchases. The data is then analyzed, and metrics are calculated. The last step is to draw a conclusion about the effectiveness of the changes by comparing the test group with the control group, which was shown the original version of the webpage.

A/B testing is considered the most precise type of marketing research. It is also the most expensive and resource-intensive.