Move using a Key Accessor very slow

Hi Alumio,

Comparable to the other topic I started about move using a pattern, I’ve now found the same for Move using a Key Accessor (but figured I’d start a new topic to keep issues separated).

In many cases you’ll have an incoming that returns an array. That’s quite annoying, since it’ll make a lot of mutations difficult. So you start by moving that array into an “array” node. I saw one incoming that took around 25 min, and figured I’d check to see what the issue was.

The issue turned out to be the Move using an Accessor. I’ve created a tester (see under) that tests the performance of that move vs. other ways to move. Just 5.000 items in an array takes around 18s.

The Move using a Pattern which was recently improved takes about 100th to 1000th of the time.

After I replaced the Move using a Key Accessor (from-key: empty, to-key: “array”) with a Move using a Pattern (from-key: *, to-key: “array.$1”), the time of the incoming went from 25 minutes to 3,5 seconds.

This is the test-config I used:
{“$schema”:“",“type”:“list-transformer”,“identifier”:“performance-move-using-accessor-tester”,“name”:"Performance Move Using Accessor Tester”,“description”:“”,“object”:{“prototype”:“data”,“parameters”:{“transformers”:[{“prototype”:“value-setter”,“parameters”:{“configurations”:[{“key”:“array”,“value”:,“mappers”:[{“prototype”:“list-fill”,“parameters”:{“start”:0,“length”:“5000”}}]}]}},{“prototype”:“list-transformer-wrapper”,“parameters”:{“transformers”:[{“prototype”:“data”,“parameters”:}]}},{“prototype”:“accessor-move”,“parameters”:{“source”:{“prototype”:“key”,“parameters”:{“root”:“array”}},“destination”:{“prototype”:“key”,“parameters”:},“stripEnclosures”:true}},{“prototype”:“list-transformer-wrapper”,“parameters”:{“transformers”:[{“prototype”:“data”,“parameters”:}]}},{“prototype”:“pattern-move”,“parameters”:{“pattern”:“*”,“replacement”:“array.$1”}},{“prototype”:“list-transformer-wrapper”,“parameters”:{“transformers”:[{“prototype”:“data”,“parameters”:}]}},{“prototype”:“key-filter”,“parameters”:{“accessor”:{“prototype”:“key”,“parameters”:}}},{“prototype”:“value-setter”,“parameters”:{“configurations”:[{“key”:“status”,“value”:“success”}]}}]}},“owner”:null,“disabled”:false}

1 Like

Hi Floris,

Thank you for your post and for taking the time to send us your valuable feedback.

We will be sure to pass your feedback on to the team for further discussion. Once we have any updates, we will get back to you.

Feel free to let us know if you have any other feedback.

Hi @floris,

would you mind exporting the sample configuration and upload it here as a file (the .ndjson)? We have difficulty in importing the JSON you provided.

Hi @Gugi ,

See attached!

performance-move-using-accessor-tester_transformer.ndjson (1.2 KB)

Hi Floris,

We have added the story to improve the transformer to our backlog. We may probably use the similar optimization like we did to Move using a pattern.

While we’re committed to implementing these changes, we’re unable to provide a specific delivery date at this time. We appreciate your understanding, and we’ll let you know once the improvement has been released.

Thank you for being part of our community.