Spotify Launches Confidence, a New Tool for Simplifying A/B Testing for Software Developers

Spotify, the popular music streaming giant, has announced the introduction of a new product for software development teams named "Confidence." It's a commercial tool based on Spotify's proprietary experimentation platform, designed to simplify the process of setting up, executing, coordinating, and analyzing user tests for software development teams. The aim is to facilitate quicker optimization of their ideas. For now, Confidence is only available as a private beta.

In a blog post, the company stated, "Our data scientists and engineers have been refining our product testing methods for several years. The objective is to ensure seamless coordination of simultaneous A/B tests or AI recommendation systems across various platforms like mobile, desktop, and web. The platform we've built scales experimentation best practices and capabilities to all our teams. Soon, any company that seeks to build, test, and iterate ideas the way we do at Spotify will have access to this platform."

According to Spotify, Confidence is ideal for software development teams either looking for an efficient way to start A/B testing or those who feel they've outgrown their current testing platform. Spotify’s journey into the world of experimentation reportedly started in the early 2010s. Initially, a handful of data scientists and engineers began conducting small-scale, internal A/B tests manually. Despite being error-prone, they recognized the value of experimentation and aimed to improve. This led to the creation of their basic A/B testing platform, ABBA, which supported feature flagging and analysis for a set of standardized metrics. The company claims this triggered a wave of experimentation across the company, with the number of priority experiments skyrocketing from fewer than 20 to hundreds per year across multiple squads.

Confidence will be available in three formats to customers. It can be accessed as a managed service, meaning the experimentation platform can be used as a standalone web service managed by Spotify. Alternatively, it can be run as a Backstage plugin or integrated into your own infrastructure via APIs. Software development teams can now sign up for the waitlist to receive an invite to Confidence. However, Spotify has not yet disclosed when Confidence will be made more widely available.

This isn’t Spotify’s first foray into offering commercial products aimed at monetizing developers. Last December, Spotify launched its plan to monetize developers through its open-source Backstage project. Backstage is a platform designed to streamline companies' infrastructure by enabling them to build customized “developer portals," combining all their tooling, apps, data, services, APIs, and documents in a single interface. This new announcement signals that, while Spotify is primarily known for its consumer-facing music streaming service, the company is exploring new avenues beyond this domain.