Adapting to Bubble's New Pricing: Utilizing AWS, Hookdeck & Tinybird to reduce your WU usage.
How to use platforms like AWS API Gateway, Hookdeck and Tinybird to offload your app's most compute and data intensive operations and reduce your Bubble WU usage.
If you’re anything like me you’ve been closely following the fallout from the big changes that were announced to Bubble’s pricing last week.
In this issue, I’m going to cover the pricing update at a high level, and some of the mitigation strategies I had already implemented in my own app to increase speed & reduce cost.
Bubble Wokflow Unit Based Pricing
It's been almost a year since we first heard about upcoming changes to Bubble's pricing model. Now that the changes have been officially announced, there's a lot to digest.
The most significant update is that Bubble will now price its services based on "workflow units" (WUs) consumed by your app, transitioning away from the previous capacity-based model.
Upon the initial announcement, numerous high-usage apps faced the prospect of staggering price increases. In my own case, I was looking at an approximate tenfold hike in my current pricing.
Fortunately, earlier this week, Bubble adjusted the way WUs are calculated, resulting in more manageable price increases for many of us.
While it's true that some (myself included) are still facing considerable pricing hikes, I recognize the immense value that Bubble offers as a platform for app development, and how it’s reduced the overall cost of bringing my product to market.
As Bubble's co-founder, Josh, mentioned in the forum, this new pricing model will enable the company to make significant additions to their engineering team, aiming to double it every year as a result of these pricing changes. This should translate to an even better experience for all of us.
Looking ahead, I remain hopeful that WU pricing will decrease as backend infrastructure continues to improve. This would mirror the progress seen with OpenAI's GPT-3.5-Turbo API, which offered a 10x cost reduction compared to its previous iteration.
Optimizing Your App with External Services to Reduce WU Usage in Bubble
I should preface this section with a note that WU optimization in Bubble is still extremely fresh - and I still have a lot to learn about optimizing my own application. All the strategies I outline below are things I had already implemented for performance reasons - but also have helped reduce my WU utilization under this new system.
As UserLoop has grown over the past year, I've been focusing on migrating some of our most data and compute-intensive operations away from Bubble. Initially, this was driven by a desire to improve speed and customer experience.
However, with the recent updates to Bubble's pricing, offloading these operations will also help ensure the platform remains cost-effective for us in the long term.
By shifting some of our most resource-intensive tasks to external services, we can still enjoy the benefits of building software in Bubble, such as rapid iteration, a user-friendly interface builder, and convenient workflows. This approach also enables us to:
Manage large volumes of data inflows
Offer a fast and comprehensive API capable of handling millions of queries
Execute quick data queries, even when dealing with millions of rows
There are a few critical areas within our app that demand substantial processing power. We've successfully migrated these functions to third-party platforms while maintaining a strong connection to Bubble for the app's user interface:
Our survey widget, embedded into e-commerce checkout flows, requires millions of API calls to fetch configuration data and submit responses back to our backend.
We process millions of real-time webhooks from various e-commerce platforms.
Our analytics feature needs to be fast and responsive, even when handling millions of records.
UserLoop Data Backend
Originally, these functions were executed natively within Bubble. While performance was the primary reason for migrating them, the new WU-based pricing makes it even more practical for app developers to consider offloading resource-intensive operations outside of Bubble.
In the following sections, we'll explore the key components of our technology stack and how they seamlessly integrate with Bubble to optimize performance and reduce WU consumption.
AWS API Gateway
Our surveys, embedded on third-party websites, need to make millions of API calls to our Bubble backend for configuration retrieval and data submission. To achieve lightning-fast response times, we've implemented AWS API Gateway in front of our custom Bubble endpoints. I covered how to set this up in this previous issue.
This integration not only accelerates request processing by over 10x (response times through AWS average 30ms, compared to 3 seconds or more when connecting directly to Bubble), but it also reduces calls made to the Bubble backend by 99% with its caching layer.
AWS queries the Bubble backend periodically (or when we explicitly clear the cache) and stores a cached version of the request. This cache is then used to serve subsequent requests without directly accessing Bubble.
As a result, we achieve significant savings by reducing the need for Bubble to run workflows or fetch data frequently. In most cases, the response is served directly from the AWS API Gateway cache, optimizing our app's performance and minimizing Bubble's workflow unit consumption.
Hookdeck
We use a powerful service called Hookdeck to manage our incoming webhooks, which offers several advantages when handling a high volume of webhooks:
Routing webhooks to multiple destinations
Filtering webhooks based on customizable rules
Handling retries if the destination is unavailable
Transforming webhooks into new data structures
Implementing HMAC security authorization
Adding security tokens in headers for secure Bubble backend calls
Directing webhooks to multiple destinations as needed
These features allow us to avoid sending all webhooks directly to a Bubble backend workflow. Instead, we can route them to other services and transform the data as required.
We employ Hookdeck to consume all our webhook data and then forward it to our datastore, Tinybird (which we'll discuss in the next section). This approach enables us to ingest large amounts of data without directly involving Bubble, optimizing performance and reducing the platform's workload.
Tinybird
The latest addition to our technology stack is Tinybird, a robust, large-scale datastore that powers all of our analytics. Tinybird is designed to store millions of records and execute rapid queries, making it an ideal solution for operations that can be slow and costly in Bubble.
Incorporating Tinybird as our primary datastore has enabled us to offload a significant portion of data storage and processing from Bubble. This not only reduces the workload on the platform but also enhances our app's overall performance.
To communicate with Tinybird, we use their straightforward REST API. This allows us to efficiently store and retrieve the data we need, streamlining our app's data interactions.
Key benefits of integrating Tinybird with Bubble:
Scalability: Tinybird can handle large volumes of data, making it suitable for apps that require extensive data storage and processing capabilities.
Performance: Its high-speed query execution feature dramatically improves the performance of data-intensive applications, ensuring a smooth user experience.
Cost-efficiency: By offloading data storage and processing to Tinybird, you can reduce the resource consumption in Bubble, potentially lowering costs associated with workflow units.
Flexibility: Tinybird's REST API simplifies data storage and retrieval, making it easier to manage large amounts of records within your app.
Everything is through the API Connector. Tinybird’s UI gives you example API requests ready for you to paste right into the Bubble API connector. This works for both storing and querying the data, so it helps you get set up super quick.
The only thing to bare in mind with Tinybird is you need to learn the intracies of how the platform works. It’s based on a database called Clickhouse and requires you to write some SQL queries to build your API endpoints.
Thankfully the Tinybird UI helps you with a lot of autocomplete when writing your queries, and you can also get ChatGPT to help you with creating them - it works amazingly well!!
That’s it for this week!
I wanted to give you a brief overview of some of the things I’ve been thinking about in my own application to reduce my workflow usage over the next 18 months. I’m planning on doing in-depth issues of this newsletter, and video tutorials for AWS API Gateway, Hookdeck and Tinybird.
Each of them is a powerful platform that plays extremely well with Bubble, but there’s a lot to them and they really command an entire edition of the newsletter each to go into all the details of how to set them up and get the most out of them - so stay tuned for those!
I hope you’ve found this issue helpful - do drop me a reply or find me on Twitter @jamesdevonport to give me your feedback.
Great article James. Its great to understand how others are using bubble at scale
To be honest, this seems crazy.
Folks go to bubble for a low code all in one tool to get something built and out to the market.
One price change and makers are adding multiple layers of complexity just to ‘get around’ it.
You now have to manage multiple layers of potential causes of an issues. You need expertise in each app to be sure you are getting the most out of their pricing plans and services..
If a price change causes this… I think you might be ready to hire developers and think long term about the app?