firefox network timeline

January 09 2023 12:53 pm

Drastically Improved Response Time Autoscaling

There is value in being able to define a desired response time range and to autoscale to keep an application in that metric range.

Heroku add-on 123 Dyno offers an alternative, definable, and consistent option to autoscale applications by their response times for maximum control with critical options like Path Blocklisting long running urls, choice of average, maximum, or percentile metric values, and choice of sample window.

Heroku stock autoscaling does its job but can be inconsistent, uncontrollable, and strangely defined due to the fact that it is depdendent on the past days M95 response time value. It does however exceed in simplicity, able to be setup in a few button clicks. Lets take a look at the below example for a Performance dynos set to autoscale with a 500ms target with a max number of dynos at 3, a mostly blank instance with no M95 history.


The standard autoscaling jumps to 2 when M95 threshold is exceeded but does not autoscale further given higher response time values simulating increased load. If response time autoscaling is returning up to 10x what the target is, the application should autoscale. This can be inconsistent due to the nature of M95 being calculated over the past day's values along with other factors. Downscaling also can take minutes even when no requests are being sent to a server to act as guard rails for potential future load.

123 Dyno addresses this with the concept of strictly defined & configurable response time Goldilocks ranges, upscaling when response times exceed defined range and downscaling when response times fall below defined range with a 12x speed boost to accomodate burst traffic. The add-on allows you to define exactly what you need and have it happen. 123 Dyno understands that applications are nuanced and gives you the full toolkit to make autoscaling work for you.

See the configuration below for 5 second upscaling intervals on M95 between 1 and 3 dynos autoscale exactly to specification.

As you can see in the autoscaling logs and response times 123 Dyno autoscales exactly to spec when it leaves it's Goldilocks range every 5 seconds under increasing simulated response time load of 800ms. Afterwards it autoscales down every minute as specified given M95 of the past 2 minute window.

Accurate Response Time Values: Pruning Paths From Autoscaling Calculation

Accurate response time data is important for accurate, efficient autoscaling. Long running values triggering autoscaling when they shouldn't and altering response time calculation of M95 over the past sample window. 123 Dyno gives you the ability to prune URLs from autoscaling calculation along with configurable sample window.

See below the Strapi Headless CMS instance this blog is running on the, /upload path takes up to 32000ms to complete for large files. A day of uploading large files or other long running paths can either throw off M95 calculation or trigger autoscaling increasing your bill. 123 Dyno offers the analytics you see above to notice this and a URL Blocklist to prune out long running paths in your application, saving you money from inaccurate autoscale events.

Response Time Autoscaling That Works

Altoghether these options allow for accurate response time autoscaling that works consistently and is not offered anywhere else except with 123 Dyno on Heroku. Have your service well defined in a response time range and autoscale to remain in range.

What else can I do with 123 Dyno?

If response time autoscaling isn't a good fit for you application, 123 Dyno extends Heroku with all the options you need to handle load. Autoscale via CPU, memory and queue times for both web and worker dynos up to 12x faster. 123 Dyno was designed to provide an autoscaling extension that provides everything you need to intelligently autoscale an application for any occasion, more use cases below.

Are your response times to variable, look into autoscaling by CPU.

Need a speed increase? Adjustable autoscaling intervals on 123 Dyno can provide an up to 12x speed boost.