Table of contents
- Introduction
- Reducing the level of detail in log files
- Reducing the number of tasks exported per execution
- Reducing the number of scheduled jobs run in parallel
- Upgrading your Alumio environment
Introduction
This article will provide insights into various ways to optimize your Alumio iPaaS environment. This may be needed to reduce the load on your environment and guarantee the stability of the various integrations.
Reducing the level of detail in log files
Alumio gives users the possibility to enable logging in various stages of the process. By default, Alumio will log all middleware-related actions. This can not be disabled and will always be logged in full detail. Examples of these logs are:
- Filtered entities
- Created tasks
- Alumio process-related information
On top of these logs, Alumio gives users the ability to enable logging for the systems they integrate. Think of authentication logs, API logs, database logs, webhook logs, etc. These logs are very useful for debugging any issues related to the systems integrated. However, in some cases, these logs can cause lots of load on the Alumio environment. Examples of when this might happen are:
- A lot of logs are being created. I.e. Alumio pulls 100k products and does an API call for each product to retrieve the stock and price from another system
- The log files are extremely big
To prevent this from causing issues it’s advised to stick to the Simple and Long logging types. While Full seems very useful, it is advised to only use this during the testing phase. As the name already suggests, Full, will store the complete requests and responses. Simple will only store the header information while Long will store up to 10.000 characters, which should be sufficient to debug any issues that may arise.
Reducing the number of tasks exported per execution
Most integrations built using Alumio’s iPaaS are asynchronous. This means that the data is being synchronized in the background and that there are two steps to each integration: filling the queue and processing the queue. In some cases you may opt to go with a real-time processing integration, essentially reducing the process to a single step: filling the queue and immediately processing it. Depending on the number of tasks processed this may cause a high load on your Alumio environment depending on the complexity of your integration. Some integrations require additional API calls to complete the data object, or each task might contain a lot of information. Such integrations could lead to high usage of server resources when a high number of tasks is processed within each execution.
It is advised to reduce this number by optimizing your scheduled jobs, reducing the number of entities send per webhook, or introducing batch processing.
Reducing the number of scheduled jobs run in parallel
Sometimes an Alumio environment has been configured to start many scheduled jobs in parallel. This could lead to a high load on your Alumio environment. For this reason, it is advised to spread the number of scheduled jobs run across the hour. Please read our article on optimizing your scheduled jobs.
Upgrading your Alumio environment
In most cases, following the tips given above should help you keep your Alumio environment within acceptable load levels. When this is not the case, it is best to consider upgrading your Alumio environment. This can be actioned by reaching out to the support team.