Combining dataloaders¶

class metatensor.models.utils.data.combine_dataloaders.CombinedDataLoader(dataloaders: List[DataLoader], shuffle: bool)[source]¶

Combines multiple dataloaders into a single dataloader.

This is useful for learning from multiple datasets at the same time, each of which may have different batch sizes, properties, etc.

Parameters:

dataloaders (List[DataLoader]) – list of dataloaders to combine
shuffle (bool) – whether to shuffle the combined dataloader (this does not act on the individual batches, but it shuffles the order in which they are returned)

Returns:

the combined dataloader

Returns the total number of batches in all dataloaders.

This returns the total number of batches in all dataloaders (as opposed to the total number of samples or the number of individual dataloaders).

__dict__ = mappingproxy({'__module__': 'metatensor.models.utils.data.combine_dataloaders', '__doc__': '\n Combines multiple dataloaders into a single dataloader.\n\n This is useful for learning from multiple datasets at the same time,\n each of which may have different batch sizes, properties, etc.\n\n :param dataloaders: list of dataloaders to combine\n :param shuffle: whether to shuffle the combined dataloader (this does not\n act on the individual batches, but it shuffles the order in which\n they are returned)\n\n :return: the combined dataloader\n ', '__init__': <function CombinedDataLoader.__init__>, 'reset': <function CombinedDataLoader.reset>, '__iter__': <function CombinedDataLoader.__iter__>, '__next__': <function CombinedDataLoader.__next__>, '__len__': <function CombinedDataLoader.__len__>, '__dict__': <attribute '__dict__' of 'CombinedDataLoader' objects>, '__weakref__': <attribute '__weakref__' of 'CombinedDataLoader' objects>, '__annotations__': {}})¶