Skip to content

Commit

Permalink
Docs: Update no code experiments (#10377)
Browse files Browse the repository at this point in the history 
Co-authored-by: Daniel Bachhuber <[email protected]>
  • Loading branch information
@ivanagas @danielbachhuber
ivanagas and danielbachhuber authored Jan 17, 2025
1 parent 1226121 commit a037b63
Show file tree
Hide file tree
Showing 4 changed files with 52 additions and 105 deletions.
2 changes: 1 addition & 1 deletion contents/blog/best-optimizely-alternatives.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ The core of feature flags and experimentation are similar between the two. The b
<ComparisonRow column1={true} column2={false} feature="Self-serve" description="Free to try, no mandatory sales calls" />
<ComparisonRow column1={true} column2={true} feature="Feature flags" description="Deploy features safely with targeting and percentage rollouts" />
<ComparisonRow column1={true} column2={true} feature="A/B testing" description="Run multivariate tests and see the impact of changes with custom goals and reports" />
<ComparisonRow column1={false} column2={true} feature="Visual experiment editor" description="Easily create experiments that make changes to your site or app" />
<ComparisonRow column1={true} column2={true} feature="Visual experiment editor" description="Easily create experiments that make changes to your site or app" />
<ComparisonRow column1={true} column2={true} feature="Funnel tests" description="Use conversion funnels as goals for your A/B test" />
<ComparisonRow column1={true} column2={true} feature="Advanced targeting" description="Target flags and A/B tests on segments and non-users" />
<ComparisonRow column1={false} column2={true} feature="Personalization" description="Deliver targeted content and products to users" />
Expand Down
149 changes: 47 additions & 102 deletions contents/docs/experiments/no-code-web-experiments.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,126 +6,79 @@ showTitle: true

> 🚧 **Note:** No-code web experiments are currently considered in `beta`. To access them, enable the [feature preview](https://us.posthog.com#panel=feature-previews) in your PostHog account. You'll also need to define `disable_web_experiments: false` in your Posthog web snippet configuration.
>
> We are keen to gather as much feedback as possible so if you try this out please let us know. You can email [[email protected]](mailto:[email protected]), send feedback via the [in-app support panel](https://us.posthog.com#panel=support%3Afeedback%3Aexperiments%3Alow), or use one of our other [support options](/docs/support-options).
> We are keen to gather as much feedback as possible so if you try this out please let us know. You can email [[email protected]](mailto:daniel.b@posthog.com), send feedback via the [in-app support panel](https://us.posthog.com#panel=support%3Afeedback%3Aexperiments%3Alow), or use one of our other [support options](/docs/support-options).
No-code web experiments enable you to run A/B tests, multivariate tests, and other experiments that modify your website without writing a single line of code.

Build your web experiment with the PostHog [toolbar](/docs/toolbar), our no-code tool, and then use the **preview variant** button to verify that everything works as expected.
You build your web experiment with the PostHog [toolbar](/docs/toolbar), our no-code tool, and then use the **preview variant** button to verify that everything works as expected.

## How to set up a no code experiment
In this example, we will change the text of a button on a sample webpage without writing any code.

In this example, we will the font size of a button on a sample webpage without writing any code.
## Set up your experiment in PostHog

### Set up your experiment in PostHog
After enabling the **No Code Web experiments** feature preview, go to the [experiments tab](https://us.posthog.com/experiments) in PostHog and click **New experiment**. On the **new experiment** page, fill out your experiment details and select **No-code web experiment** as your experiment type, and click **Save as draft**.

Go to the [experiments tab](https://us.posthog.com/experiments) in PostHog and click **New experiment**. On the **new experiment** page, fill out your experiment detail and select your **experiment type**.
<ProductScreenshot
imageLight="https://res.cloudinary.com/dmukukwp6/image/upload/Clean_Shot_2025_01_16_at_13_47_05_2x_b727914878.png"
imageDark="https://res.cloudinary.com/dmukukwp6/image/upload/Clean_Shot_2025_01_16_at_13_46_40_2x_3d54ac114d.png"
alt="Experiment in PostHog"
classes="rounded"
/>

The **experiment type** setting enables you to choose how your experiment is implemented. Note that once the experiment type is selected and saved, it cannot be changed.
> **What about product experiments?** [Product experiments](/docs/experiments/creating-an-experiment) use custom code to manage how variants modify your product. This option is best if you need specific, code-based adjustments and configurations.
![Select experiment type](https://res.cloudinary.com/dmukukwp6/image/upload/create_web_exp_type_ed2e6af988.png)
## Implement the experiment on your website

#### Experiment types

1. **Product experiment**
- Choose this option if you intend to use custom code to manage how variants modify your product.
- Recommended for experiments that require specific, code-based adjustments and configurations.

2. **No-code web experiment**
- Choose this option to define variants on your website without writing code.
- This setup leverages the PostHog toolbar, allowing you to make adjustments entirely within your browser.

We will select the **no-code web experiment** option for our example project.


### Implement the experiment on your website

Once you've created the no-code web experiment, you are brought to the **Implementation** section of your experiment on PostHog.
This is where you can define variant changes directly on your website using the PostHog toolbar. The toolbar enables you to select elements and apply transformations for each variant in your experiment. To use it:
> **Note:** During the beta period, you'll need to add `disable_web_experiments: false` to your PostHog web snippet configuration like this:
>
> ```js-web
> posthog.init('<ph_project_api_key>', {
> api_host: '<ph_client_api_host>',
> disable_web_experiments: false,
> })
> ```
> Otherwise, you'll be able to set up the experiment in the toolbar but it won't actually run on your live site.
1. Click the **Launch toolbar on your website** button.
Once you've created your no-code web experiment, click **Launch toolbar on your website** under the **Implementation** section.
![Implement web experiment](https://res.cloudinary.com/dmukukwp6/image/upload/web_exp_implementation_ddd1848103.png)
2. Click the **vial** tab on the toolbar and use it to select the elements you want to modify for each variant.

3. Apply the desired transformations to implement the changes.

The toolbar will guide you through the process of setting up the variant changes visually on your site, making it easier to customize without coding.

![Toolbar launched](https://res.cloudinary.com/dmukukwp6/image/upload/create_web_exp_toolbar_launched_3b23c18d3f.png)


### How to edit an experiment variant

Experiment variants enable you to implement and test changes to specific elements on your webpage. For example, you can test different call-to-action (CTA) button sizes and evaluate their effectiveness compared to the baseline (control group).

The toolbar enables you to select elements and apply transformations for each variant in your experiment. Once you launch it:
#### 1. View and edit variants
1. Click the **vial** tab on the toolbar.
On the experiment detail page, each variant is listed with its rollout percentage. For example:
- **control**: `rollout: 33%`
- **larger_cta**: `rollout: 33%`
- **smaller_cta**: `rollout: 34%`
2. Select the variant you want to modify the site for. In our case, we select the **test** variant.
You can click on a variant to view or edit it:
- Review the variant name (e.g., `larger_cta`).
- If no modifications are defined yet, a message such as _"This experiment variant doesn't modify any elements"_ will appear.
- Use the options to add or remove elements.
3. Click **Add element** and select the element you want to modify. We select the main signup button on our page.
#### 2. Add or modify elements
4. Use the toolbar to modify the element's text, CSS, or HTML. We'll change the button's text to say "Sign up for free". Importantly, the **Text** field modifies the text of the element. The **CSS** field lets you change the styling of the element. The **HTML** field will override the entire element.
5. If you'd like to change more parts of the page for this version of the experiment, click **Add element** to modify a second, third, or fourth element.
To make changes to a variant:
- Click **Add element** to include specific webpage elements (e.g., buttons, headers, images) that you want to modify.
- Define the modifications (e.g., increase button size, change text color).
6. Once done modifying elements for your variants, click **Save experiment**.
#### 3. Add new variants
![Implementing web experiments with the toolbar](https://res.cloudinary.com/dmukukwp6/image/upload/Clean_Shot_2025_01_16_at_13_56_01_2x_aff05d4224.png)
To create a new variant:
- Click the **Add variant** button at the bottom of the editor.
- Enter a name for the new variant.
- Add and configure elements for the new variant.
### Further editing your experiment
#### 4. Save changes
The toolbar also enables further editing of your experiment. For example, you can:
Once all modifications are complete:
- Click the **Save experiment** button to apply the changes.
- Add multiple element changes to a variant by clicking **+ Add element**. Elements are buttons, headers, images, and more.
- Remove an element from a variant by clicking the trash bin icon.
- Add more versions of your experiment by clicking **+ Add variant**.
- Change all elements, headers, buttons, and/or images by clicking the **Change element** dropdown.
#### 5. Cancel edits (optional)
## How to see how your experiment will look
To discard any changes and return to the previous view, click **Cancel**.
To preview the different versions of your experiment, go back to your experiment details page in PostHog and click the **Variants** tab. Here, you'll see your variants along with a button to preview each of them (and the ability to add a screenshot for them).
<ProductScreenshot
imageLight="https://res.cloudinary.com/dmukukwp6/image/upload/Clean_Shot_2025_01_16_at_14_06_28_2x_1ca848f7ca.png"
imageDark="https://res.cloudinary.com/dmukukwp6/image/upload/Clean_Shot_2025_01_16_at_14_06_40_2x_8acab7c726.png"
alt="Preview variants in PostHog"
classes="rounded"
/>
### How to modify elements in an experiment variant

![Select HTML element](https://res.cloudinary.com/dmukukwp6/image/upload/create_web_exp_select_element_75c46a1ed3.png)

Open the experiment editor in the toolbar and select the variant you want to modify (e.g., `test`).

Next, click the **Select element** dropdown in the variant editor. This provides options like:
- **All elements**: Apply changes globally to all relevant elements on the page.
- **Headers**: Modify header text or style.
- **Buttons**: Customize button properties such as size, color, or text.
- **Images**: Adjust image properties like dimensions or alt text.

After selecting an element, define the specific changes for the selected element:
- **HTML**: Adjust the underlying HTML structure or attributes.
- **Text**: Update the displayed text content for the selected element.
- **CSS**: Modify visual styling, such as colors, font size, padding, or margins.

Any selected element in your experiment's variant can have multiple modifications applied to it.

To modify additional elements, click **Add element** and repeat the selection process.

Finally, once all desired modifications are applied, click **Save experiment** to finalize the updates.


### How to preview a variant

Previewing variants allows you to verify their configuration and ensure that the changes are displayed and functioning as expected before the experiment goes live.

To do so, first, locate the variant you want to preview in the **distribution** section of the experiment editor (e.g., `control`, `larger_cta`, or `smaller_cta`).

Once you find it, click the **Preview variant** button next to the desired variant. This opens a live preview of the variant’s user interface, enabling you to see the applied changes in action.
Doing this enables you to verify their configuration and ensure that the changes are displayed and functioning as expected before the experiment goes live.
### Final touches
Expand All @@ -134,13 +87,5 @@ Congratulations! You're now ready to launch a web experiment.
Before launching, ensure the following:
1. **Test thoroughly**: Verify that all variants display and function as expected across devices and browsers. See our [testing feature flags doc](/docs/feature-flags/testing) for more details.
2. **Document changes**: Record the modifications made to each variant and their goals.

Also, during the beta period, you'll need to add `disable_web_experiments: false` to your PostHog web snippet configuration, e.g.:
```js-web
posthog.init('<ph_project_api_key>', {
api_host: '<ph_client_api_host>',
disable_web_experiments: false,
})
```
2. **Document changes**: Record the modifications made to each variant and their goals. Add a description of the changes to your experiment as well as screenshots of each of the variants.
2 changes: 1 addition & 1 deletion contents/handbook/growth/sales/why-buy-posthog.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ With disparate tools, it's hard to get an understanding of your users both indiv
- After launching an experiment, click to see the session replays of users experiencing a specific variant, showing you whether your users are experiencing it as intended.
- Create a survey tied to an experiment (via the feature flag), letting you gather qualitative feedback from users who are experiencing this new feature for the first time.
- Attach revenue data stored in our data warehouse from Stripe to product analytics to see behaviors which drive revenue acquisition and growth.
- Use Heatmaps to identify potential dead zones on your website, roll out a no-code experiment with your conversion funnel as a goal.
- Use Heatmaps to identify potential dead zones on your website, roll out a [no-code experiment](/docs/experiments/no-code-web-experiments) with your conversion funnel as a goal.
## Consolidated pricing

As we have a low price point and can help replace a number of disparate tools they may be running customers are highly likely to save versus the cost of running multiple tools in parallel:
Expand Down
4 changes: 3 additions & 1 deletion contents/tutorials/webflow-ab-tests.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Optimizing your marketing site often requires testing small changes against each

This tutorial shows how to set up A/B tests with Webflow and PostHog to get the most out of your marketing site.

> Thanks to Julia at [Mayan](https://www.mayan.co/) for inspiring this tutorial.
> **Looking for a no-code option for A/B testing?** Check out our recently released [no-code web experiments](/docs/experiments/no-code-web-experiments) feature.
## Setup Webflow and PostHog

Expand Down Expand Up @@ -69,6 +69,8 @@ Once you launch this experiment, PostHog automatically tracks the results and ma

Now that you set up an A/B test in Webflow with PostHog, you can use this same pattern to run experiments and A/B tests throughout your site. Happy testing!

> Thanks to Julia at [Mayan](https://www.mayan.co/) for inspiring this tutorial.
## Further reading

- [How to set up Webflow analytics and session recordings](/tutorials/webflow)
Expand Down

0 comments on commit a037b63

Please sign in to comment.