Challenge

When I joined Fred. Olsen Cruise Lines, there was no structured way to validate design and product decisions before they shipped. Changes went live based on best judgement rather than evidence, with no consistent way to measure impact, learn from what didn’t work, or build a case for future investment. If I wanted UX decisions — mine or anyone else’s — to be trusted and prioritised, I needed to build the capability to prove they worked, not just argue for them.

Process

• Securing buy-in: I presented experimentation as a key lever within my initial UX strategy, building the case for why the business needed it before asking for budget.
  • Platform selection: I researched experimentation platforms and led the initiation of sales calls with vendors, shortlisting to three, then briefed an RFP to formalise requirements and compare them fairly. I negotiated contracts and selected Optimizely, starting with their Web Experimentation platform.
  • Onboarding: I managed the onboarding process myself, making sure the right stakeholders were involved and the platform was set up and implemented correctly from day one.

• Governance first: I set up a RACI to clarify ownership across e-commerce, product, and development, so experimentation had clear accountability.
  • A hypothesis log: I built a central log to capture, structure, and track test ideas from across the business, rather than relying on ad-hoc requests.
  • A prioritisation framework: I introduced a consistent way to score and rank ideas by potential impact and effort, so the programme’s limited capacity went on the tests most likely to matter.
• A test dashboard and learning library: I built a dashboard to track progress of all test plans, live tests, and historical test results, with a roadmap view for future planning. Paired with learnings and insights, and structured with tags and categories for easy filtering, ensuring insights and learnings are documented and used continuously in future hypotheses.
• Robust documentation: Every step of the process is documented and captured, including test plans, QA requirements, and release notices shared with stakeholders.

Proving the model

• I ran the first tests myself — including the search tool and search dropdown redesigns — end-to-end, to demonstrate the value of the process before asking for further investment.

Embedding the mindset

• I hosted workshops across the wider e-commerce team to build a shared understanding of experimentation and design thinking, rather than keeping it as specialist knowledge.

Scaling delivery

As demand grew beyond what I could deliver alone, I brought in and now manage an agency partnership to increase test throughput, while retaining ownership of strategy, prioritisation, and quality.

Solution

A structured, sustainable experimentation programme with:

Results

Key takeaways

• Building the governance and prioritisation model before scaling delivery meant growth didn’t come at the cost of quality or focus.
• Running the first tests myself gave the programme credibility — people trust a process once they’ve seen it work, not before.
• Workshops that spread the mindset across the wider team multiplied the programme’s impact well beyond what I could have delivered alone.

Managing an agency partnership let me scale test throughput while staying the strategic owner of what gets tested and why.