MultimodalPlaceRecognitionTrainer

A module that implements a training algorithm for a multimodal neural network model of global localization based on the contrastive learning approach.

Usage example

You should initialize the opr.trainers.place_recognition.multimodal.MultimodalPlaceRecognitionTrainer class with the desired parameters:

from opr.trainers.place_recognition.multimodal import MultimodalPlaceRecognitionTrainer

trainer = MultimodalPlaceRecognitionTrainer(
    modalities_weights=modalities_weights,  # dictionary like {"image": 1.0, "cloud": 1.0}
    checkpoints_dir=checkpoints_dir,
    model=model,
    loss_fn=loss_fn,
    optimizer=optimizer,
    scheduler=scheduler,
    batch_expansion_threshold=batch_expansion_threshold,  # value in range [0, 1]
    wandb_log=True,  # or False
    device="cuda",
)

To start training, you should call the opr.trainers.place_recognition.multimodal.MultimodalPlaceRecognitionTrainer.train() method:

trainer.train(
    epochs=100,
    train_dataloader=train_dataloader,
    val_dataloader=train_dataloader,
    test_dataloader=train_dataloader,
)

If you want to test the model on a test dataset, you should call the opr.trainers.place_recognition.multimodal.MultimodalPlaceRecognitionTrainer.test() method:

trainer.test(test_dataloader)

More usage examples can be found in the following scripts and notebooks: