We use HTTP clients on systems A and B. I have got products from A and stored the incoming entities in Product-storage and can “Filter previous”. But say now next time we run the product incoming, we get some updates. I can not create them again, our system B endpoint is create only. I can make an outgoing config for update/put, but can I conditionally choose the endpoint/outgoing to use for this? So for the same data, when new (does not exist on B) do POST and when CHANGED do PUT?
@kjetil you can use a HTTP publisher/transformer as outgoing configuration as this takes parameters as input for the URL, method & payload.
You can use a entity transformer to determine whether it’s a new product or an existing one. Depending on the result, you can dynamically set the URL, method & payload parameters.
My suggested approach would be to perform the following steps:
-
Create a storage that will hold existing products. I’d suggest to set the TTL to 1 day so every day the product is rechecked (if required).
-
I’d create a outgoing configuration that uses the Storage publisher to publish the flag to this storage once the request has been performed and the product has been created or updated.
-
On each subsequent task check the storage made in step #1 to see if the flag is set. If it isn’t set, perform a request to system B to check whether it exists (fallback scenario). If it does exist, set flag in storage so it can be used for the next update for same product.
-
Use a conditional transformer to set the URL, request and payload variables which then can be used in the HTTP transformer where you are going to do the actual request.
-
Perform the request using a HTTP transformer and using the set variables.
I created a quick sample (non-working) configuration that demonstrates above idea.
export_20230509094548.ndjson (2.3 KB)
I tested a lookup with a merge-http transformer. For existing product it’s ok. However the non-existing product returns 404 header together with a message (product not found). The log on testing the transformer shows this as not successfull instead of just not setting the existing-product. Will this still work inside an outgoing config, or will it break, meaning no non-existing products will ever be created (since the lookup ended in error)?
In the HTTP client used you can set HTTP error ranges.
If you remove 404 from the range it should continue processing