Are there any tools to manage research data especially designed for Machine Learning?

Question

In my work, I deal with different type of data and try to develop machine learning models to learn relationship within that data. My data consist of a mix of images, tabular data or even text data. During my work I usually deal with issues, where my data runs through different processing pipelines. These pipelines on one hand can take a while (up to several hours) and also renames the files. For my work, I aim to guarantee reproducibility and also share my data.

Therefore, my question: Are there any tools that help machine learning practitioners to simplify there

Answer 1

For organizing research date we have developed Kadi4Mat (https://kadi.iam.kit.edu/) which helps you to organize your date and enable them to be used with in conjunction different ML/AI methods (https://kadi.iam.kit.edu/kadi-ai). Check out the website to find more cited work how this can be applied to various applications.

Answer 2

For dealing with research software, there is archetype Betty:
https://nfdi4ing.de/archetypes/betty/

Still, there is no best practice on how to manage data in the context of AI applications.
You would have to document the inputs, used software and outputs of every step to make it reproducible.

All the best
Tobias