Customer Experience Assessment

Bowlmor/AMF corporation, with over 300 locations across the US and 10,000 employees, retained Studio28 to assess the customer experience (CX) practice at the corporate level. We analyzed the performance of their enterprise feedback management vendor, the validity of solicited Net Promoter Score surveys (100,000 yearly) and the reliability of the algorithm used to classify unsolicited feedback (online comments) from Yelp and Facebook.

What we did

  • Performed a detailed assessment of how surveys, secret shopper reports and online sentiment data were integrated into operational decision-making.
  • Interviewed location managers and line staff on how their operations and internal systems impact their ability to utilize CX feedback in their daily work.
  • Thoroughly assessed the statistical validity and methodology behind surveys and secret shopper reports and checked the reliability online sentiment analysis.

Findings

    • Corporate managers and staff found the CX feedback interesting but often of questionable value. Users disliked the interface, system dashboards were irrelevant to them and the majority never logged in to view reports.

    • Methodological errors in the survey questions made the results and their impact on key performance indicators (KPI’s) unreliable.

    • The artificial intelligence algorithm used to analyze the sentiment of the natural language in survey responses and online comments often misclassified the results and generated faulty recommendations.

    • When asked, the vendor was unaware of these issues, agreed that they were important and committed to working with the client to fix them.

Results

    • The client asked Studio28 to evaluate alternative vendors and present a shortlist of potential replacements.

    • Studio28 gathered and documented detailed requirements for the enterprise CX vendor candidates, researched alternative vendors and narrowed the candidates down to four.

    • We then prepared a request for information (RFI) for vendor candidates and arranged for the presentation of system demos.