How to convert Sortformer Diarizer using nemo2riva #74
Description
I have tried to convert a custom fine-tuned sortformer model from nemo to riva
But i got the error, this is the log:
nemo2riva --out diar_sortformer_4spk-v1.riva --onnx-opset=19 diar_sortformer_4spk-v1.nemo
fused_indices_to_multihot has reached end of life. Please migrate to a non-experimental function.
OneLogger: Setting error_handling_strategy to DISABLE_QUIETLY_AND_REPORT_METRIC_ERROR for rank (rank=0) with OneLogger disabled. To override: explicitly set error_handling_strategy parameter.
No exporters were provided. This means that no telemetry data will be collected.
[NeMo W 2026-01-19 13:28:36 warnings:109] /usr/local/lib/python3.10/site-packages/pydub/utils.py:170: RuntimeWarning: Couldn't find ffmpeg or avconv - defaulting to ffmpeg, but may not work
warn("Couldn't find ffmpeg or avconv - defaulting to ffmpeg, but may not work", RuntimeWarning)
[NeMo I 2026-01-19 13:28:37 nemo2riva:45] Logging level set to 20
[NeMo I 2026-01-19 13:28:37 nemo2riva:49] Restoring NeMo model from 'diar_sortformer_4spk-v1.nemo'
INFO: GPU available: True (cuda), used: True
INFO: TPU available: False, using: 0 TPU cores
INFO: HPU available: False, using: 0 HPUs
INFO: Trainer(limit_train_batches=1.0) was configured so 100% of the batches per epoch will be used..
INFO: Trainer(limit_val_batches=1.0) was configured so 100% of the batches will be used..
INFO: Trainer(limit_test_batches=1.0) was configured so 100% of the batches will be used..
INFO: Trainer(limit_predict_batches=1.0) was configured so 100% of the batches will be used..
INFO: Trainer(val_check_interval=1.0) was configured so validation will run at the end of the training epoch..
[NeMo W 2026-01-19 13:28:39 sortformer_diar_models:94] If you intend to do training or fine-tuning, please call the ModelPT.setup_training_data() method and provide a valid configuration file to setup the train data loader.
Train config :
manifest_filepath: null
sample_rate: 16000
num_spks: 4
session_len_sec: 90
soft_label_thres: 0.5
soft_targets: false
labels: null
batch_size: 4
shuffle: true
num_workers: 18
validation_mode: false
use_lhotse: false
use_bucketing: false
num_buckets: 10
bucket_duration_bins:
- 10
- 20
- 30
- 40
- 50
- 60
- 70
- 80
- 90
pin_memory: true
min_duration: 80
max_duration: 90
batch_duration: 400
quadratic_duration: 1200
bucket_buffer_size: 20000
shuffle_buffer_size: 10000
window_stride: 0.01
subsampling_factor: 8
[NeMo W 2026-01-19 13:28:39 sortformer_diar_models:94] If you intend to do validation, please call the ModelPT.setup_validation_data() or ModelPT.setup_multiple_validation_data() method and provide a valid configuration file to setup the validation data loader(s).
Validation config :
manifest_filepath: null
is_tarred: false
tarred_audio_filepaths: null
sample_rate: 16000
num_spks: 4
session_len_sec: 90
soft_label_thres: 0.5
soft_targets: false
labels: null
batch_size: 4
shuffle: false
num_workers: 18
validation_mode: true
use_lhotse: false
use_bucketing: false
drop_last: false
pin_memory: true
window_stride: 0.01
subsampling_factor: 8
[NeMo W 2026-01-19 13:28:39 sortformer_diar_models:94] Please call the ModelPT.setup_test_data() or ModelPT.setup_multiple_test_data() method and provide a valid configuration file to setup the test data loader(s).
Test config :
manifest_filepath: null
is_tarred: false
tarred_audio_filepaths: null
sample_rate: 16000
num_spks: 4
session_len_sec: 90
soft_label_thres: 0.5
soft_targets: false
labels: null
batch_size: 4
shuffle: false
seq_eval_mode: true
num_workers: 18
validation_mode: true
use_lhotse: false
use_bucketing: false
drop_last: false
pin_memory: true
window_stride: 0.01
subsampling_factor: 8
[NeMo I 2026-01-19 13:28:41 modelPT:501] Model SortformerEncLabelModel was successfully restored from /data/diar_sortformer_4spk-v1.nemo.
[NeMo I 2026-01-19 13:28:41 schema:191] Loaded schema file /usr/local/lib/python3.10/site-packages/nemo2riva/validation_schemas/tts-exported-radttsmodel.yaml for nemo.collections.tts.models.RadTTSModel
[NeMo I 2026-01-19 13:28:41 schema:191] Loaded schema file /usr/local/lib/python3.10/site-packages/nemo2riva/validation_schemas/nlp-pc-exported-bert.yaml for nemo.collections.nlp.models.PunctuationCapitalizationModel
[NeMo I 2026-01-19 13:28:41 schema:191] Loaded schema file /usr/local/lib/python3.10/site-packages/nemo2riva/validation_schemas/tts-exported-hifiganmodel.yaml for nemo.collections.tts.models.HifiGanModel
[NeMo I 2026-01-19 13:28:41 schema:191] Loaded schema file /usr/local/lib/python3.10/site-packages/nemo2riva/validation_schemas/nlp-qa-exported-bert.yaml for nemo.collections.nlp.models.QAModel
[NeMo I 2026-01-19 13:28:42 schema:191] Loaded schema file /usr/local/lib/python3.10/site-packages/nemo2riva/validation_schemas/tts-exported-fastpitchmodel.yaml for nemo.collections.tts.models.FastPitchModel
[NeMo I 2026-01-19 13:28:42 schema:191] Loaded schema file /usr/local/lib/python3.10/site-packages/nemo2riva/validation_schemas/nlp-mt-exported-encdecmtmodel.yaml for nemo.collections.nlp.models.MTEncDecModel
[NeMo I 2026-01-19 13:28:42 schema:191] Loaded schema file /usr/local/lib/python3.10/site-packages/nemo2riva/validation_schemas/nlp-isc-exported-bert.yaml for nemo.collections.nlp.models.IntentSlotClassificationModel
[NeMo I 2026-01-19 13:28:42 schema:191] Loaded schema file /usr/local/lib/python3.10/site-packages/nemo2riva/validation_schemas/nlp-tc-exported-bert.yaml for nemo.collections.nlp.models.TextClassificationModel
[NeMo I 2026-01-19 13:28:42 schema:191] Loaded schema file /usr/local/lib/python3.10/site-packages/nemo2riva/validation_schemas/asr-stt-exported-encdecctcmodel.yaml for nemo.collections.asr.models.EncDecCTCModel
[NeMo I 2026-01-19 13:28:42 schema:191] Loaded schema file /usr/local/lib/python3.10/site-packages/nemo2riva/validation_schemas/asr-stt-exported-encdectcmodelbpe.yaml for nemo.collections.asr.models.EncDecCTCModelBPE
[NeMo I 2026-01-19 13:28:42 schema:191] Loaded schema file /usr/local/lib/python3.10/site-packages/nemo2riva/validation_schemas/nlp-tkc-exported-bert.yaml for nemo.collections.nlp.models.TokenClassificationModel
[NeMo I 2026-01-19 13:28:42 schema:191] Loaded schema file /usr/local/lib/python3.10/site-packages/nemo2riva/validation_schemas/asr-scr-exported-encdecclsmodel.yaml for nemo.collections.asr.models.classification_models.EncDecClassificationModel
[NeMo I 2026-01-19 13:28:42 schema:191] Loaded schema file /usr/local/lib/python3.10/site-packages/nemo2riva/validation_schemas/nlp-mt-exported-megatronnmtmodel.yaml for nemo.collections.nlp.models.MegatronNMTModel
[NeMo W 2026-01-19 13:28:42 convert:61] Validation schema not found for nemo.collections.asr.models.SortformerEncLabelModel.
That means Riva does not yet support a pipeline for this network and likely will not work with it.
[NeMo I 2026-01-19 13:28:42 artifacts:97] Found model at ./model_weights.ckpt
[NeMo I 2026-01-19 13:28:42 convert:83] Retrieved artifacts: dict_keys(['model_config.yaml'])
[NeMo I 2026-01-19 13:28:42 convert:87] Exporting model SortformerEncLabelModel with config=ExportConfig(export_subnet='', export_format='ONNX', export_file='model_graph.onnx', encryption=None, autocast=False, max_dim=None, export_args={})
[NeMo E 2026-01-19 13:28:42 convert:87] ERROR: Export failed. Please make sure your NeMo model class (<class 'nemo.collections.asr.models.sortformer_diar_models.SortformerEncLabelModel'>) has working export() and that you have the latest NeMo package installed with [all] dependencies.
Traceback (most recent call last):
File "/usr/local/bin/nemo2riva", line 8, in
sys.exit(nemo2riva())
File "/usr/local/lib/python3.10/site-packages/nemo2riva/cli/nemo2riva.py", line 49, in nemo2riva
Nemo2Riva(args)
File "/usr/local/lib/python3.10/site-packages/nemo2riva/convert.py", line 87, in Nemo2Riva
export_model(
File "/usr/local/lib/python3.10/site-packages/nemo2riva/cookbook.py", line 135, in export_model
raise e
File "/usr/local/lib/python3.10/site-packages/nemo2riva/cookbook.py", line 90, in export_model
_, descriptions = model.export(
File "/usr/local/lib/python3.10/site-packages/nemo/core/classes/exportable.py", line 117, in export
out, descr, out_example = model._export(
File "/usr/local/lib/python3.10/site-packages/nemo/core/classes/exportable.py", line 197, in _export
output_example = self.forward(*input_list, **input_dict)
TypeError: SortformerEncLabelModel.forward_for_export() missing 4 required positional arguments: 'spkcache', 'spkcache_lengths', 'fifo', and 'fifo_lengths'
i'm using nemo2riva 2.22.0