Extra rate limit options

[jesse.wever]

Hello everyone,
I am currently making a configuration that hits the api rate limit of the system i’m connecting to, sadly the rate limiter that Alumio offers does not work this api on since it expects an int to be given but the api response is x/40

Keen eyed will spot that this is a shopify api, and the standard shopify connectors would probably solve this issue but sadly i do not think i can use those since this call is made within a loop using a http transformer.

in short what is my request:
that the api rate limit also accepts 5/40 input

[abubakar]

You can actually use the merger transformer->http transformer(enable header response), can be utilized alongside the condition transformer to manage the limit, by including a sleep transformer

[Gugi]

Hi @jesse.wever

At the moment, we don’t support handling the Shopify rate limit using X-Shopify-Shop-Api-Call-Limit header. However, we already have a story to implement rate limit handling using the Retry-After header in our roadmap.

As a workaround, the Sleep Transformer solution from @abubakar would work for you. Another solution is to use an HTTP client plugin called Retry requests on failure. The plugin will order Alumio to retry the request when the response’s status code is between a range of numbers. Please find the below sample.

The above configuration will order Alumio to wait for 30 seconds and retry at a max. of 3 times if it receives a response with a status code of 429.

Please give them a try and let us know if you have any further questions.

[jesse.wever]

this would also work, thank you.
I will add it to my transformer

[anon99371720]

I-am not sure if it’s applicable for your issue, but @jesse posted two ideas on a similar topic.
With the addition of reading rate limit headers.

Alumio Task: Single threaded

Hi @abubakar, Within a single route Alumio should never process the tasks multi-threaded. I suppose you are running into a problem with rate limiting. You can do two things: First thing I would check is whether the API returns a header that gives information about ratelimiting. An example header would be: X-RateLimit-Minutely-Remaining: 60 This header indicates that I can send another 60 requests during this minute. With each subsequent request, the number goes down. During the next minute…

@Gugi Some extra information.

[jesse.wever]

I looked at that post before making this one, and used the sleep method as jesse described there and then combined that with the retry that gugi wrote about here.

[jesse]

@jesse.wever possibly you can optimise it a little further by conditionally letting the transformer sleep?

There’s a option to retrieve the request headers on the HTTP transformer. It’s called Add headers to response body. You can then parse the Shopify rate-limiting header and based on your condition, i.e. value equals 40/40, you could execute the Sleep transformer. This way you don’t create unneeded 429s to the Shopify API and possibly increase the throughput of your integration.