Alumio Task: Single threaded

Hello @anon99371720 and @Gugi ,
At present, I am engaged in a project that involves an API call limit of once every 10 seconds. In order to divide the process into separate components and execute them one by one, I included a “get branch by pattern” feature in the entity transformer. However, it appears that this feature uses a multi-threaded process, which is not suitable for my purposes since the API calls are restricted to once every 10 seconds. Instead, I need a single-threaded process(finish one first and start another). Unfortunately, I am unsure of how to implement this in Alumio

Hi @abubakar,

Within a single route Alumio should never process the tasks multi-threaded. I suppose you are running into a problem with rate limiting.

You can do two things:

  1. First thing I would check is whether the API returns a header that gives information about ratelimiting. An example header would be:
X-RateLimit-Minutely-Remaining: 60

This header indicates that I can send another 60 requests during this minute. With each subsequent request, the number goes down. During the next minute, the value resets to it’s original value.

In order to support this within Alumio, you need to add a plugin to the HTTP client (or specifically to the HTTP transformer/publisher where you are doing the request). The plugin is called Rate limit based on header. You can fill in the header name, the threshold when Alumio needs to pause the requests and how long it should wait for. This is something that you have to think about and optimize as you go along.

  1. Second option in case your endpoint does not reveal any rate limiting header, is to implement a ‘Sleep transformer’. You could add this immediately after the API call so that Alumio will wait 10 seconds before proceeding.

Good luck! :slight_smile:

1 Like

Thank you @jesse , I will try the first option because the second option is already in place

1 Like