Confluent > Case Studies > Storyblocks' Transformation: Redesigning Data Pipeline with Confluent

Storyblocks' Transformation: Redesigning Data Pipeline with Confluent

Confluent Logo
 Storyblocks' Transformation: Redesigning Data Pipeline with Confluent - IoT ONE Case Study
Technology Category
  • Analytics & Modeling - Machine Learning
  • Application Infrastructure & Middleware - Event-Driven Application
Applicable Industries
  • Equipment & Machinery
  • Oil & Gas
Use Cases
  • Inventory Management
  • Predictive Maintenance
Services
  • Cloud Planning, Design & Implementation Services
  • Data Science Services
The Customer
About The Customer
Storyblocks is a leading media company that provides an unlimited download subscription-based service for stock video and audio. It has over 100,000 customers in the television and video production industry, including NBC and MTV, plus tens of thousands of hobbyists looking to enhance their video projects and productions. Storyblocks’ subscribers can download an unlimited number of clips from a vast and rapidly growing library of stock video, production music, motion backgrounds, sound effects, special effects, and more. As the company transitioned from a disruptor to a major industry player, it began to experience issues with its data pipeline, which led to the partnership with Confluent.
The Challenge
Storyblocks, a rapidly growing media company, faced challenges with its monolithic application and synchronous REST API calls between services. As the company transitioned from a disruptor to a major industry player, it began to experience issues with the application they had built when the company was started. The issues persisted even after they had split the monolith into microservices. Developers and data engineers were unable to resolve issues quickly or iterate on their search functionality with sufficient agility. The increasing amount of data threatened to slow down productivity and time to market for new features. The initial solution, an AWS Kinesis data pipeline dumping raw data into Amazon S3, began to fail due to lack of scalability and suitability for fully decoupling services.
The Solution
To manage these challenges, Storyblocks needed a new solution that could form the backbone of an entirely new data pipeline. They began to use Apache Kafka® and did a proof of concept in conjunction with using a schema registry from Aiven. However, they weren't getting the support for Kafka that they needed from Aiven, so they started to use Confluent Cloud, a fully managed cloud service for Kafka. They also began to spread the use of Kafka as an event bus for more streaming applications and machine learning (ML) features. With this Confluent-backed data pipeline in place, the Storyblocks team could begin to affect a true digital transformation both internally and externally. Instead of implementing a queue for inter-services communication, the team just puts it on the pipeline where it’s stored forever. This sort of infinite storage is powerful for two reasons: Events can be replayed on demand with powerful in-built schema validation and analysts can look at historical data when answering various questions for the business.
Operational Impact
  • The implementation of Confluent Cloud has brought significant operational benefits to Storyblocks. It has reduced technical debt and total cost of ownership (TCO) from REST APIs and modernized a monolithic application. The solution has also supported the company's rebranding effort by consolidating the search for audio and visual files. Furthermore, it has vastly improved data visibility, collection, and communication for data scientists and developers. The new data pipeline has enabled the company to iterate on their search algorithm quickly, improving the user search experience. Looking forward, the company plans to use the event-based architecture to improve the user experience and tackle the lucrative enterprise market.
Quantitative Benefit
  • Reduced technical debt and total cost of ownership (TCO) from REST APIs
  • Consolidated search for audio and visual files
  • Supported rebranding effort

Case Study missing?

Start adding your own!

Register with your work email and create a new case study profile for your business.

Add New Record

Related Case Studies.

Contact us

Let's talk!
* Required
* Required
* Required
* Invalid email address
By submitting this form, you agree that IoT ONE may contact you with insights and marketing messaging.
No thanks, I don't want to receive any marketing emails from IoT ONE.
Submit

Thank you for your message!
We will contact you soon.