| cohere2_vision | multi | test_model_integration_batched_generate | OOM | 7/7 | (line 1240) torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 288.00 MiB. GPU 1 has a total capacity of 22.30 GiB of which 106.69 MiB is free. Process 84186 has 22.19 GiB memory in use. Of the allocated mem… |
| cohere2_vision | multi | test_model_integration_forward | OOM | 7/7 | (line 1240) torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 288.00 MiB. GPU 1 has a total capacity of 22.30 GiB of which 28.69 MiB is free. Process 84186 has 22.27 GiB memory in use. Of the allocated memo… |
| cwm | multi | test_cwm_generation_20_tokens | OOM | 7/7 | (line 353) torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 1.47 GiB. GPU 0 has a total capacity of 22.30 GiB of which 1.45 GiB is free. Process 395323 has 20.85 GiB memory in use. Of the allocated memory … |
| cwm | multi | test_cwm_sliding_window_long_sequence | OOM | 7/7 | (line 103) torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 52.00 MiB. GPU 1 has a total capacity of 22.30 GiB of which 42.69 MiB is free. Process 395323 has 22.25 GiB memory in use. Of the allocated memor… |
| deepseek_vl_hybrid | multi | test_model_text_generation_batched | OOM | 7/7 | (line 1374) torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 32.00 MiB. GPU 0 has a total capacity of 22.30 GiB of which 2.69 MiB is free. Process 926536 has 22.29 GiB memory in use. Of the allocated memor… |
| deepseek_vl_hybrid | multi | test_model_text_generation_with_multi_image | OOM | 5/7 | (line 857) torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 44.00 MiB. GPU 0 has a total capacity of 22.30 GiB of which 40.69 MiB is free. Process 611336 has 22.26 GiB memory in use. Of the allocated memor… |
| exaone4 | multi | test_model_generation_beyond_sliding_window | OOM | 7/7 | (line 280) torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 220.00 MiB. GPU 0 has a total capacity of 22.30 GiB of which 138.69 MiB is free. Process 726766 has 22.16 GiB memory in use. Of the allocated mem… |
| glm4_moe_lite | multi | test_compile_static_cache | OOM | 7/7 | (line 1240) torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB. GPU 0 has a total capacity of 22.30 GiB of which 16.69 MiB is free. Process 526562 has 22.28 GiB memory in use. Of the allocated memo… |
| glm4_moe_lite | single | test_compile_static_cache | OOM | 7/7 | (line 1240) torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB. GPU 0 has a total capacity of 22.30 GiB of which 16.69 MiB is free. Process 626409 has 22.28 GiB memory in use. Of the allocated memo… |
| llama4 | multi | test_model_17b_16e_batch | OOM | 7/7 | (line 958) torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 4.02 GiB. GPU 1 has a total capacity of 22.30 GiB of which 2.60 GiB is free. Process 713966 has 19.70 GiB memory in use. Of the allocated memory … |
| llama4 | single | test_model_17b_16e_batch | OOM | 7/7 | (line 958) torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 4.02 GiB. GPU 0 has a total capacity of 22.30 GiB of which 3.07 GiB is free. Process 433148 has 19.22 GiB memory in use. Of the allocated memory … |
| llama4 | single | test_model_17b_16e_fp32 | OOM | 7/7 | (line 353) torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 2.50 GiB. GPU 0 has a total capacity of 22.30 GiB of which 2.23 GiB is free. Process 433148 has 20.06 GiB memory in use. Of the allocated memory … |
| mamba2 | multi | test_batched_equivalence_without_cache | OOM | 6/7 | (line 1374) torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 64.00 MiB. GPU 0 has a total capacity of 22.30 GiB of which 26.69 MiB is free. Process 751419 has 22.27 GiB memory in use. Of the allocated memo… |
| mamba2 | multi | test_simple_generate | OOM | 7/7 | (line 1374) torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 256.00 MiB. GPU 0 has a total capacity of 22.30 GiB of which 28.69 MiB is free. Process 751419 has 22.27 GiB memory in use. Of the allocated mem… |
| mamba2 | single | test_batched_equivalence_without_cache | OOM | 6/7 | (line 1374) torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 64.00 MiB. GPU 0 has a total capacity of 22.30 GiB of which 26.69 MiB is free. Process 284022 has 22.27 GiB memory in use. Of the allocated memo… |
| mamba2 | single | test_simple_generate | OOM | 7/7 | (line 1374) torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 64.00 MiB. GPU 0 has a total capacity of 22.30 GiB of which 28.69 MiB is free. Process 618194 has 22.27 GiB memory in use. Of the allocated memo… |
| qwen3_vl_moe | multi | test_small_model_integration_test_with_video | OOM | 7/7 | (line 1240) torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 768.00 MiB. GPU 1 has a total capacity of 22.30 GiB of which 358.69 MiB is free. Process 127736 has 21.95 GiB memory in use. Of the allocated me… |
| zamba | multi | test_simple_batched_generate_with_padding | OOM | 7/7 | (line 1374) torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 228.00 MiB. GPU 0 has a total capacity of 22.30 GiB of which 12.69 MiB is free. Process 612705 has 22.28 GiB memory in use. Of the allocated mem… |
| zamba | single | test_simple_batched_generate_with_padding | OOM | 7/7 | (line 1374) torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 228.00 MiB. GPU 0 has a total capacity of 22.30 GiB of which 12.69 MiB is free. Process 835377 has 22.28 GiB memory in use. Of the allocated mem… |
| zamba | single | test_simple_generate | OOM | 7/7 | (line 1374) torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 228.00 MiB. GPU 0 has a total capacity of 22.30 GiB of which 12.69 MiB is free. Process 835377 has 22.28 GiB memory in use. Of the allocated mem… |
| generation | multi | test_matches_immediate_check_at_max_length | cuda_runtime | 7/7 | (line 1374) torch.AcceleratorError: CUDA error: device-side assert triggered |
| generation | multi | test_matches_immediate_check_on_a_batch | cuda_runtime | 7/7 | (line 1374) torch.AcceleratorError: CUDA error: device-side assert triggered |
| generation | multi | test_matches_immediate_check_when_stopping_early | cuda_runtime | 7/7 | (line 1374) torch.AcceleratorError: CUDA error: device-side assert triggered |
| generation | multi | test_matches_immediate_check_with_a_sliding_window | cuda_runtime | 7/7 | (line 1374) torch.AcceleratorError: CUDA error: device-side assert triggered |
| generation | multi | test_matches_immediate_check_without_a_cache | cuda_runtime | 7/7 | (line 1374) torch.AcceleratorError: CUDA error: device-side assert triggered |
| generation | multi | test_validate_assistant | cuda_runtime | 7/7 | (line 1920) torch.AcceleratorError: CUDA error: device-side assert triggered |
| generation | single | test_matches_immediate_check_at_max_length | cuda_runtime | 7/7 | (line 1374) torch.AcceleratorError: CUDA error: device-side assert triggered |
| generation | single | test_matches_immediate_check_on_a_batch | cuda_runtime | 7/7 | (line 1374) torch.AcceleratorError: CUDA error: device-side assert triggered |
| generation | single | test_matches_immediate_check_when_stopping_early | cuda_runtime | 7/7 | (line 1374) torch.AcceleratorError: CUDA error: device-side assert triggered |
| generation | single | test_matches_immediate_check_with_a_sliding_window | cuda_runtime | 7/7 | (line 1374) torch.AcceleratorError: CUDA error: device-side assert triggered |
| generation | single | test_matches_immediate_check_without_a_cache | cuda_runtime | 7/7 | (line 1374) torch.AcceleratorError: CUDA error: device-side assert triggered |
| generation | single | test_validate_assistant | cuda_runtime | 7/7 | (line 1920) torch.AcceleratorError: CUDA error: device-side assert triggered |
| blt | multi | test_model_logits | import_or_config | 7/7 | (line 312) AttributeError: 'BltConfig' object has no attribute 'num_hidden_layers' |
| blt | multi | test_model_logits_bf16 | import_or_config | 7/7 | (line 312) AttributeError: 'BltConfig' object has no attribute 'num_hidden_layers' |
| blt | single | test_model_logits | import_or_config | 7/7 | (line 312) AttributeError: 'BltConfig' object has no attribute 'num_hidden_layers' |
| blt | single | test_model_logits_bf16 | import_or_config | 7/7 | (line 312) AttributeError: 'BltConfig' object has no attribute 'num_hidden_layers' |
| deepseek_v32 | multi | test_batched_generation_padding | load_error | 7/7 | (line 531) ValueError: The current `device_map` had weights offloaded to the disk, which needed to be re-saved. This is either because the weights are not in `safetensors` format, or because the model uses an internal … |
| deepseek_v32 | multi | test_deepseek_v32 | load_error | 7/7 | (line 531) ValueError: The current `device_map` had weights offloaded to the disk, which needed to be re-saved. This is either because the weights are not in `safetensors` format, or because the model uses an internal … |
| deepseek_v32 | multi | test_logits_eager | load_error | 7/7 | (line 531) ValueError: The current `device_map` had weights offloaded to the disk, which needed to be re-saved. This is either because the weights are not in `safetensors` format, or because the model uses an internal … |
| deepseek_v32 | multi | test_logits_long_context | load_error | 7/7 | (line 531) ValueError: The current `device_map` had weights offloaded to the disk, which needed to be re-saved. This is either because the weights are not in `safetensors` format, or because the model uses an internal … |
| deepseek_v32 | single | test_batched_generation_padding | load_error | 7/7 | (line 531) ValueError: The current `device_map` had weights offloaded to the disk, which needed to be re-saved. This is either because the weights are not in `safetensors` format, or because the model uses an internal … |
| deepseek_v32 | single | test_deepseek_v32 | load_error | 7/7 | (line 531) ValueError: The current `device_map` had weights offloaded to the disk, which needed to be re-saved. This is either because the weights are not in `safetensors` format, or because the model uses an internal … |
| deepseek_v32 | single | test_logits_eager | load_error | 7/7 | (line 531) ValueError: The current `device_map` had weights offloaded to the disk, which needed to be re-saved. This is either because the weights are not in `safetensors` format, or because the model uses an internal … |
| deepseek_v32 | single | test_logits_long_context | load_error | 7/7 | (line 531) ValueError: The current `device_map` had weights offloaded to the disk, which needed to be re-saved. This is either because the weights are not in `safetensors` format, or because the model uses an internal … |
| deepseek_v4 | multi | test_v4_flash_dequantized_chat_seven_prompts | load_error | 7/7 | (line 531) ValueError: The current `device_map` had weights offloaded to the disk, which needed to be re-saved. This is either because the weights are not in `safetensors` format, or because the model uses an internal … |
| deepseek_v4 | multi | test_v4_flash_dequantized_generation | load_error | 7/7 | (line 531) ValueError: The current `device_map` had weights offloaded to the disk, which needed to be re-saved. This is either because the weights are not in `safetensors` format, or because the model uses an internal … |
| deepseek_v4 | single | test_v4_flash_dequantized_chat_seven_prompts | load_error | 7/7 | (line 531) ValueError: The current `device_map` had weights offloaded to the disk, which needed to be re-saved. This is either because the weights are not in `safetensors` format, or because the model uses an internal … |
| deepseek_v4 | single | test_v4_flash_dequantized_generation | load_error | 7/7 | (line 531) ValueError: The current `device_map` had weights offloaded to the disk, which needed to be re-saved. This is either because the weights are not in `safetensors` format, or because the model uses an internal … |
| minimax_m3_vl | multi | test_batched_image_generation | load_error | 7/7 | (line 531) ValueError: The current `device_map` had weights offloaded to the disk, which needed to be re-saved. This is either because the weights are not in `safetensors` format, or because the model uses an internal … |
| minimax_m3_vl | multi | test_image_and_text_generation | load_error | 7/7 | (line 531) ValueError: The current `device_map` had weights offloaded to the disk, which needed to be re-saved. This is either because the weights are not in `safetensors` format, or because the model uses an internal … |
| minimax_m3_vl | multi | test_long_context_needle_token_match | load_error | 7/7 | (line 531) ValueError: The current `device_map` had weights offloaded to the disk, which needed to be re-saved. This is either because the weights are not in `safetensors` format, or because the model uses an internal … |
| minimax_m3_vl | multi | test_padding_sides_text_and_image | load_error | 7/7 | (line 531) ValueError: The current `device_map` had weights offloaded to the disk, which needed to be re-saved. This is either because the weights are not in `safetensors` format, or because the model uses an internal … |
| minimax_m3_vl | multi | test_real_image_apple_recognition | load_error | 7/7 | (line 531) ValueError: The current `device_map` had weights offloaded to the disk, which needed to be re-saved. This is either because the weights are not in `safetensors` format, or because the model uses an internal … |
| minimax_m3_vl | multi | test_video_generation | load_error | 7/7 | (line 531) ValueError: The current `device_map` had weights offloaded to the disk, which needed to be re-saved. This is either because the weights are not in `safetensors` format, or because the model uses an internal … |
| minimax_m3_vl | single | test_batched_image_generation | load_error | 7/7 | (line 531) ValueError: The current `device_map` had weights offloaded to the disk, which needed to be re-saved. This is either because the weights are not in `safetensors` format, or because the model uses an internal … |
| minimax_m3_vl | single | test_image_and_text_generation | load_error | 7/7 | (line 531) ValueError: The current `device_map` had weights offloaded to the disk, which needed to be re-saved. This is either because the weights are not in `safetensors` format, or because the model uses an internal … |
| minimax_m3_vl | single | test_long_context_needle_token_match | load_error | 7/7 | (line 531) ValueError: The current `device_map` had weights offloaded to the disk, which needed to be re-saved. This is either because the weights are not in `safetensors` format, or because the model uses an internal … |
| minimax_m3_vl | single | test_padding_sides_text_and_image | load_error | 7/7 | (line 531) ValueError: The current `device_map` had weights offloaded to the disk, which needed to be re-saved. This is either because the weights are not in `safetensors` format, or because the model uses an internal … |
| minimax_m3_vl | single | test_real_image_apple_recognition | load_error | 7/7 | (line 531) ValueError: The current `device_map` had weights offloaded to the disk, which needed to be re-saved. This is either because the weights are not in `safetensors` format, or because the model uses an internal … |
| minimax_m3_vl | single | test_video_generation | load_error | 7/7 | (line 531) ValueError: The current `device_map` had weights offloaded to the disk, which needed to be re-saved. This is either because the weights are not in `safetensors` format, or because the model uses an internal … |
| mistral4 | multi | test_mistral_small_4_generation | load_error | 7/7 | (line 531) ValueError: The current `device_map` had weights offloaded to the disk, which needed to be re-saved. This is either because the weights are not in `safetensors` format, or because the model uses an internal … |
| mistral4 | multi | test_mistral_small_4_logits | load_error | 7/7 | (line 531) ValueError: The current `device_map` had weights offloaded to the disk, which needed to be re-saved. This is either because the weights are not in `safetensors` format, or because the model uses an internal … |
| mistral4 | single | test_mistral_small_4_generation | load_error | 7/7 | (line 531) ValueError: The current `device_map` had weights offloaded to the disk, which needed to be re-saved. This is either because the weights are not in `safetensors` format, or because the model uses an internal … |
| mistral4 | single | test_mistral_small_4_logits | load_error | 7/7 | (line 531) ValueError: The current `device_map` had weights offloaded to the disk, which needed to be re-saved. This is either because the weights are not in `safetensors` format, or because the model uses an internal … |
| qwen3_moe | multi | test_model_15b_a2b_generation | load_error | 7/7 | (line 74) ValueError: Some modules are dispatched on the CPU or the disk. Make sure you have enough GPU RAM to fit the quantized model. If you want to dispatch the model on the CPU or the disk while keeping these modul… |
| qwen3_moe | multi | test_model_15b_a2b_logits | load_error | 7/7 | (line 74) ValueError: Some modules are dispatched on the CPU or the disk. Make sure you have enough GPU RAM to fit the quantized model. If you want to dispatch the model on the CPU or the disk while keeping these modul… |
| qwen3_moe | multi | test_model_15b_a2b_long_prompt_sdpa | load_error | 7/7 | (line 74) ValueError: Some modules are dispatched on the CPU or the disk. Make sure you have enough GPU RAM to fit the quantized model. If you want to dispatch the model on the CPU or the disk while keeping these modul… |
| qwen3_moe | multi | test_speculative_generation | load_error | 7/7 | (line 74) ValueError: Some modules are dispatched on the CPU or the disk. Make sure you have enough GPU RAM to fit the quantized model. If you want to dispatch the model on the CPU or the disk while keeping these modul… |
| bridgetower | multi | test_constrastive_learning | other | 7/7 | (line 716) ValueError: Unrecognized image processor in BridgeTower/bridgetower-large-itm-mlm-itc. Should have a `image_processor_type` key in its preprocessor_config.json of config.json, or one of the following `model_… |
| bridgetower | multi | test_image_and_text_retrieval | other | 7/7 | (line 716) ValueError: Unrecognized image processor in BridgeTower/bridgetower-base-itm-mlm. Should have a `image_processor_type` key in its preprocessor_config.json of config.json, or one of the following `model_type`… |
| bridgetower | multi | test_masked_language_modeling | other | 7/7 | (line 716) ValueError: Unrecognized image processor in BridgeTower/bridgetower-base-itm-mlm. Should have a `image_processor_type` key in its preprocessor_config.json of config.json, or one of the following `model_type`… |
| bridgetower | single | test_constrastive_learning | other | 7/7 | (line 716) ValueError: Unrecognized image processor in BridgeTower/bridgetower-large-itm-mlm-itc. Should have a `image_processor_type` key in its preprocessor_config.json of config.json, or one of the following `model_… |
| bridgetower | single | test_image_and_text_retrieval | other | 7/7 | (line 716) ValueError: Unrecognized image processor in BridgeTower/bridgetower-base-itm-mlm. Should have a `image_processor_type` key in its preprocessor_config.json of config.json, or one of the following `model_type`… |
| bridgetower | single | test_masked_language_modeling | other | 7/7 | (line 716) ValueError: Unrecognized image processor in BridgeTower/bridgetower-base-itm-mlm. Should have a `image_processor_type` key in its preprocessor_config.json of config.json, or one of the following `model_type`… |
| cohere2_vision | multi | test_model_forward_vision | other | 7/7 | (line 595) OSError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/root/repos/moe/engines/command_a+_bf16'. Use `repo_type` argument if needed. |
| cohere2_vision | multi | test_model_generate_vision | other | 7/7 | (line 595) OSError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/root/repos/moe/engines/command_a+_bf16'. Use `repo_type` argument if needed. |
| cohere2_vision | single | test_model_forward_vision | other | 7/7 | (line 595) OSError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/root/repos/moe/engines/command_a+_bf16'. Use `repo_type` argument if needed. |
| cohere2_vision | single | test_model_generate_vision | other | 7/7 | (line 595) OSError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/root/repos/moe/engines/command_a+_bf16'. Use `repo_type` argument if needed. |
| colqwen2 | multi | test_model_integration_test | other | 7/7 | (line 114) TypeError: only a single or a list of entries is supported but got type=<class 'datasets.arrow_dataset.Column'> |
| colqwen2 | single | test_model_integration_test | other | 7/7 | (line 114) TypeError: only a single or a list of entries is supported but got type=<class 'datasets.arrow_dataset.Column'> |
| deepseek_vl_hybrid | multi | test_model_text_generation | other | 7/7 | (line 67) RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cuda:1! |
| deepseek_vl_hybrid | single | test_model_text_generation_with_multi_image | other | 7/7 | (line 470) RuntimeError: You can't move a model that has some modules offloaded to cpu or disk. |
| generation | multi | test_cache_device_map_with_vision_layer_device_map | other | 7/7 | (line 1644) ValueError: The device_map provided does not give any device for the following parameters: model.vision_tower.embeddings.patch_embedding.weight, model.vision_tower.embeddings.patch_embedding.bias, model.vis… |
| generation | multi | test_generate_multi_accelerator_causal_mask | other | 7/7 | (line 1644) ValueError: The device_map provided does not give any device for the following parameters: model.visual.patch_embed.proj.weight, model.visual.blocks.0.norm1.weight, model.visual.blocks.0.norm1.bias, model.v… |
| generation | single | test_cache_device_map_with_vision_layer_device_map | other | 7/7 | (line 1644) ValueError: The device_map provided does not give any device for the following parameters: model.vision_tower.embeddings.patch_embedding.weight, model.vision_tower.embeddings.patch_embedding.bias, model.vis… |
| glm_image | multi | test_image_to_image_generation | other | 7/7 | (line 322) RuntimeError: The size of tensor a (864) must match the size of tensor b (630) at non-singleton dimension 0 |
| glm_image | single | test_image_to_image_generation | other | 7/7 | (line 322) RuntimeError: The size of tensor a (864) must match the size of tensor b (630) at non-singleton dimension 0 |
| janus | multi | test_model_generate_images | other | 7/7 | (line 1247) TypeError: GenerationMixin._prepare_static_cache() missing 1 required positional argument: 'prefill_chunk_size' |
| janus | multi | test_model_text_generation | other | 7/7 | (line 2118) ValueError: Image features and image tokens do not match, tokens: 0, features: 1179648 |
| janus | multi | test_model_text_generation_with_multi_image | other | 7/7 | (line 2118) ValueError: Image features and image tokens do not match, tokens: 0, features: 2359296 |
| janus | single | test_model_generate_images | other | 7/7 | (line 1247) TypeError: GenerationMixin._prepare_static_cache() missing 1 required positional argument: 'prefill_chunk_size' |
| janus | single | test_model_text_generation | other | 7/7 | (line 2118) ValueError: Image features and image tokens do not match, tokens: 0, features: 1179648 |
| janus | single | test_model_text_generation_with_multi_image | other | 7/7 | (line 2118) ValueError: Image features and image tokens do not match, tokens: 0, features: 2359296 |
| kimi_k25 | multi | test_model_logits | other | 7/7 | (line 552) RuntimeError: indices should be either on cpu or on the same device as the indexed tensor (cuda:1) |
| kimi_k25 | multi | test_model_logits_batched | other | 7/7 | (line 552) RuntimeError: indices should be either on cpu or on the same device as the indexed tensor (cuda:1) |
| minicpm3 | multi | test_minicpm3_4b_generation | other | 7/7 | (line 199) KeyError: 'factor' |
| minicpm3 | multi | test_minicpm3_4b_logits | other | 7/7 | (line 199) KeyError: 'factor' |
| minicpm3 | single | test_minicpm3_4b_generation | other | 7/7 | (line 199) KeyError: 'factor' |
| minicpm3 | single | test_minicpm3_4b_logits | other | 7/7 | (line 199) KeyError: 'factor' |
| minicpmv4_6 | multi | test_small_model_video_generation | other | 7/7 | (line 384) RuntimeError: shape '[49, 1014, 1152]' is invalid for input of size 57286656 |
| minicpmv4_6 | single | test_small_model_video_generation | other | 7/7 | (line 384) RuntimeError: shape '[49, 1014, 1152]' is invalid for input of size 57286656 |
| peft_integration | multi | test_hotswap_with_compile_and_higher_rank_works | other | 7/7 | (line 316) RuntimeError: You set `ignore_mismatched_sizes` to `False`, thus raising an error. For details look at the above report! |
| peft_integration | multi | test_hotswap_with_compile_and_lower_rank_works | other | 7/7 | (line 316) RuntimeError: You set `ignore_mismatched_sizes` to `False`, thus raising an error. For details look at the above report! |
| peft_integration | multi | test_hotswap_without_compile_and_with_higher_rank_works | other | 7/7 | (line 316) RuntimeError: You set `ignore_mismatched_sizes` to `False`, thus raising an error. For details look at the above report! |
| peft_integration | multi | test_hotswap_without_compile_and_with_lower_rank_works | other | 7/7 | (line 316) RuntimeError: You set `ignore_mismatched_sizes` to `False`, thus raising an error. For details look at the above report! |
| peft_integration | single | test_hotswap_with_compile_and_higher_rank_works | other | 7/7 | (line 316) RuntimeError: You set `ignore_mismatched_sizes` to `False`, thus raising an error. For details look at the above report! |
| peft_integration | single | test_hotswap_with_compile_and_lower_rank_works | other | 7/7 | (line 316) RuntimeError: You set `ignore_mismatched_sizes` to `False`, thus raising an error. For details look at the above report! |
| peft_integration | single | test_hotswap_without_compile_and_with_higher_rank_works | other | 7/7 | (line 316) RuntimeError: You set `ignore_mismatched_sizes` to `False`, thus raising an error. For details look at the above report! |
| peft_integration | single | test_hotswap_without_compile_and_with_lower_rank_works | other | 7/7 | (line 316) RuntimeError: You set `ignore_mismatched_sizes` to `False`, thus raising an error. For details look at the above report! |
| pegasus | multi | test_pegasus_xsum_summary | other | 7/7 | (line 350) assert torch.Size([2, 422]) == (2, 421) |
| pegasus | single | test_pegasus_xsum_summary | other | 7/7 | (line 350) assert torch.Size([2, 422]) == (2, 421) |
| phi3 | multi | test_export_static_cache | other | 7/7 | (line 1507) torch._dynamo.exc.Unsupported: Data-dependent branching |
| phi3 | single | test_export_static_cache | other | 7/7 | (line 1507) torch._dynamo.exc.Unsupported: Data-dependent branching |
| qwen2_moe | multi | test_speculative_generation | other | 7/7 | (line 311) RuntimeError: We encountered some issues during automatic conversion of the weights. For details look at the `CONVERSION` entries of the above report! |
| qwen2_moe | single | test_speculative_generation | other | 7/7 | (line 311) RuntimeError: We encountered some issues during automatic conversion of the weights. For details look at the `CONVERSION` entries of the above report! |
| qwen3_vl_moe | multi | test_small_model_integration_test | other | 7/7 | (line 311) RuntimeError: We encountered some issues during automatic conversion of the weights. For details look at the `CONVERSION` entries of the above report! |
| qwen3_vl_moe | multi | test_small_model_integration_test_batch | other | 7/7 | (line 311) RuntimeError: We encountered some issues during automatic conversion of the weights. For details look at the `CONVERSION` entries of the above report! |
| qwen3_vl_moe | multi | test_small_model_integration_test_batch_different_resolutions | other | 7/7 | (line 311) RuntimeError: We encountered some issues during automatic conversion of the weights. For details look at the `CONVERSION` entries of the above report! |
| qwen3_vl_moe | multi | test_small_model_integration_test_batch_wo_image | other | 7/7 | (line 311) RuntimeError: We encountered some issues during automatic conversion of the weights. For details look at the `CONVERSION` entries of the above report! |
| qwen3_vl_moe | multi | test_small_model_integration_test_expand | other | 7/7 | (line 311) RuntimeError: We encountered some issues during automatic conversion of the weights. For details look at the `CONVERSION` entries of the above report! |
| qwen3_vl_moe | multi | test_small_model_integration_test_expand_with_video | other | 7/7 | (line 311) RuntimeError: We encountered some issues during automatic conversion of the weights. For details look at the `CONVERSION` entries of the above report! |
| recurrent_gemma | multi | test_model_2b_8bit | other | 7/7 | (line 158) RuntimeError: (*bias): last dimension must be contiguous |
| recurrent_gemma | single | test_model_2b_8bit | other | 7/7 | (line 158) RuntimeError: (*bias): last dimension must be contiguous |
| seamless_m4t | multi | test_speech_to_speech_model | other | 7/7 | (line 421) ValueError: Invalid input type. Must be a single audio or a list of audio |
| seamless_m4t | multi | test_speech_to_text_model | other | 7/7 | (line 421) ValueError: Invalid input type. Must be a single audio or a list of audio |
| seamless_m4t | multi | test_to_rus_speech | other | 7/7 | (line 421) ValueError: Invalid input type. Must be a single audio or a list of audio |
| seamless_m4t | single | test_speech_to_speech_model | other | 7/7 | (line 421) ValueError: Invalid input type. Must be a single audio or a list of audio |
| seamless_m4t | single | test_speech_to_text_model | other | 7/7 | (line 421) ValueError: Invalid input type. Must be a single audio or a list of audio |
| seamless_m4t | single | test_to_rus_speech | other | 7/7 | (line 421) ValueError: Invalid input type. Must be a single audio or a list of audio |
| utils | multi | test_chunked_prefill_initializes_static_cache_eagerly | other | 7/7 | (line 331) TypeError: Pointer argument must be either uint64 or have data_ptr method |
| utils | multi | test_data_parallel_dynamic_cache | other | 7/7 | (line 1834) RuntimeError: a Tensor with 2 elements cannot be converted to Scalar |
| utils | single | test_chunked_prefill_initializes_static_cache_eagerly | other | 7/7 | (line 331) TypeError: Pointer argument must be either uint64 or have data_ptr method |
| bamba | multi | test_simple_batched_generate_with_padding | output_mismatch | 7/7 | (line 627) AssertionError: '<|be[35 chars]on this lovely evening? I hope you are all doing well. Today I' != '<|be[35 chars]on this lovely evening? I hope you are doing well. I am here' |
| bamba | multi | test_simple_generate | output_mismatch | 7/7 | (line 575) AssertionError: '<|be[35 chars]on this lovely evening? I hope you are all doing well. Today I' != '<|be[35 chars]on this lovely evening? I hope you are all having a good time.' |
| bamba | single | test_simple_batched_generate_with_padding | output_mismatch | 7/7 | (line 627) AssertionError: '<|be[35 chars]on this lovely evening? I hope you are all doing well. Today I' != '<|be[35 chars]on this lovely evening? I hope you are doing well. I am here' |
| bamba | single | test_simple_generate | output_mismatch | 7/7 | (line 575) AssertionError: '<|be[35 chars]on this lovely evening? I hope you are all doing well. Today I' != '<|be[35 chars]on this lovely evening? I hope you are all having a good time.' |
| big_bird | multi | test_fill_mask | output_mismatch | 7/7 | (line 921) AssertionError: '' != 'happiness' |
| big_bird | single | test_fill_mask | output_mismatch | 7/7 | (line 921) AssertionError: '' != 'happiness' |
| bitnet | multi | test_model_logits | output_mismatch | 7/7 | (line 198) AssertionError: Tensor-likes are not close! |
| bitnet | single | test_model_logits | output_mismatch | 7/7 | (line 198) AssertionError: Tensor-likes are not close! |
| blip_2 | multi | test_inference_t5 | output_mismatch | 7/7 | (line 1658) AssertionError: Lists differ: [0, 2335, 1556, 28, 1782, 30, 8, 2608, 1] != [0, 3, 9, 2335, 19, 1556, 28, 160, 1782, 30, 8, 2608, 1] |
| blip_2 | multi | test_inference_t5_batched_beam_search | output_mismatch | 7/7 | (line 1713) AssertionError: Lists differ: [0, 2335, 1556, 28, 1782, 30, 8, 2608, 1] != [0, 3, 9, 2335, 19, 1556, 28, 160, 1782, 30, 8, 2608, 1] |
| blip_2 | multi | test_inference_t5_multi_accelerator | output_mismatch | 7/7 | (line 1782) AssertionError: Lists differ: [0, 2335, 1556, 28, 1782, 30, 8, 2608, 1] != [0, 3, 9, 2335, 19, 1556, 28, 160, 1782, 30, 8, 2608, 1] |
| blip_2 | single | test_inference_t5 | output_mismatch | 7/7 | (line 1658) AssertionError: Lists differ: [0, 2335, 1556, 28, 1782, 30, 8, 2608, 1] != [0, 3, 9, 2335, 19, 1556, 28, 160, 1782, 30, 8, 2608, 1] |
| blip_2 | single | test_inference_t5_batched_beam_search | output_mismatch | 7/7 | (line 1713) AssertionError: Lists differ: [0, 2335, 1556, 28, 1782, 30, 8, 2608, 1] != [0, 3, 9, 2335, 19, 1556, 28, 160, 1782, 30, 8, 2608, 1] |
| bloom | multi | test_batch_generated_text | output_mismatch | 7/7 | (line 621) AssertionError: Lists differ: ['Hello what is', 'Running a quick test with the followi[54 chars]the'] != ['Hello what is the best way to get the data from the se[127 chars]on2'] |
| bloom | multi | test_batch_generation_padding | output_mismatch | 7/7 | (line 586) AssertionError: Lists differ: [5941[15 chars]632, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,[82 chars]0, 0] != [5941[15 chars]632, 419, 682, 15, 473, 912, 267, 40704, 15, 1[186 chars] 912] |
| bloom | multi | test_simple_generation | output_mismatch | 7/7 | (line 539) AssertionError: 'I en[58 chars] play. I am a very active person, and I am a v[75 chars]am a' != 'I en[58 chars] play with the kids. I am a very active person[86 chars]nd I' |
| bloom | single | test_batch_generated_text | output_mismatch | 7/7 | (line 621) AssertionError: Lists differ: ['Hello what is', 'Running a quick test with the followi[54 chars]the'] != ['Hello what is the best way to get the data from the se[127 chars]on2'] |
| bloom | single | test_batch_generation_padding | output_mismatch | 7/7 | (line 586) AssertionError: Lists differ: [5941[15 chars]632, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,[82 chars]0, 0] != [5941[15 chars]632, 419, 682, 15, 473, 912, 267, 40704, 15, 1[186 chars] 912] |
| bloom | single | test_simple_generation | output_mismatch | 7/7 | (line 539) AssertionError: 'I en[58 chars] play. I am a very active person, and I am a v[75 chars]am a' != 'I en[58 chars] play with the kids. I am a very active person[86 chars]nd I' |
| cohere2 | multi | test_model_pipeline_bf16 | output_mismatch | 7/7 | (line 160) AssertionError: 'Hell[22 chars] for my school and I need to create a website [31 chars] the' != 'Hell[22 chars] for a school assignment and I need to create [37 chars]have' |
| cohere2 | single | test_model_pipeline_bf16 | output_mismatch | 7/7 | (line 160) AssertionError: 'Hell[22 chars] for my school and I need to create a website [31 chars] the' != 'Hell[22 chars] for a school assignment and I need to create [37 chars]have' |
| cohere2_vision | single | test_model_integration_forward | output_mismatch | 7/7 | (line 687) AssertionError: False is not true : Actual logits: tensor([2.3711, 1.6689, 1.8389, 1.9785, 1.9131], dtype=torch.float16) |
| colqwen2 | multi | test_model_integration_test_2 | output_mismatch | 7/7 | (line 405) AssertionError: Expected scores tensor([[16.3750, 10.9375, 14.7500], |
| colqwen2 | single | test_model_integration_test_2 | output_mismatch | 7/7 | (line 405) AssertionError: Expected scores tensor([[16.3750, 10.9375, 14.7500], |
| convnextv2 | multi | test_inference_image_classification_head | output_mismatch | 7/7 | (line 315) AssertionError: Tensor-likes are not close! |
| convnextv2 | single | test_inference_image_classification_head | output_mismatch | 7/7 | (line 315) AssertionError: Tensor-likes are not close! |
| cvt | multi | test_inference_image_classification_head | output_mismatch | 7/7 | (line 278) AssertionError: Tensor-likes are not close! |
| cvt | single | test_inference_image_classification_head | output_mismatch | 7/7 | (line 278) AssertionError: Tensor-likes are not close! |
| dab_detr | multi | test_inference_object_detection_head | output_mismatch | 7/7 | (line 800) AssertionError: Tensor-likes are not close! |
| dab_detr | single | test_inference_object_detection_head | output_mismatch | 7/7 | (line 800) AssertionError: Tensor-likes are not close! |
| deepseek_vl | multi | test_model_text_generation_batched | output_mismatch | 7/7 | (line 149) AssertionError: Lists differ: ['You[222 chars]tant:The image depicts a snowy landscape with [367 chars]the'] != ['You[222 chars]tant:What is a cat, a cat, a cat, a cat, a cat[329 chars]the'] |
| deepseek_vl | single | test_model_text_generation_batched | output_mismatch | 7/7 | (line 149) AssertionError: Lists differ: ['You[222 chars]tant:The image depicts a snowy landscape with [367 chars]the'] != ['You[222 chars]tant:What is a cat, a cat, a cat, a cat, a cat[329 chars]the'] |
| deepseek_vl_hybrid | single | test_model_text_generation_batched | output_mismatch | 7/7 | (line 377) AssertionError: Lists differ: ['You[224 chars]nt:\nThe image depicts a fluffy, light brown a[371 chars]he '] != ['You[224 chars]nt:\nA fluffy animal in a fluffyThe image,The [329 chars]he '] |
| dia | multi | test_dia_model_integration_generate_audio_context | output_mismatch | 7/7 | (line 738) AssertionError: Tensor-likes are not equal! |
| dia | single | test_dia_model_integration_generate_audio_context | output_mismatch | 7/7 | (line 738) AssertionError: Tensor-likes are not equal! |
| diffllama | multi | test_compile_static_cache | output_mismatch | 7/7 | (line 484) AssertionError: Lists differ: ['Sim[41 chars]that 1) the speed of light is constant in all [301 chars]y p'] != ['Sim[41 chars]that 2.5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 '[133 chars]a a'] |
| diffllama | single | test_compile_static_cache | output_mismatch | 7/7 | (line 484) AssertionError: Lists differ: ['Sim[41 chars]that 1) the speed of light is constant in all [301 chars]y p'] != ['Sim[41 chars]that 2.5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 '[133 chars]a a'] |
| diffusion_gemma | multi | test_diffusion_gemma_chat_template | output_mismatch | 7/7 | (line 897) AssertionError: Lists differ: [[2, [46 chars]23613, 236761, 106, 107, 105, 4368, 107]] != [[2, [46 chars]23613, 236761, 106, 107, 105, 4368, 107, 100, 45518, 107, 101]] |
| diffusion_gemma | multi | test_minified_diffusion_gemma_forward_batched | output_mismatch | 7/7 | (line 1694) AssertionError: Tensor-likes are not close! |
| diffusion_gemma | multi | test_minified_diffusion_gemma_forward_text_only | output_mismatch | 7/7 | (line 1507) AssertionError: Tensor-likes are not close! |
| diffusion_gemma | multi | test_minified_diffusion_gemma_forward_with_image | output_mismatch | 7/7 | (line 1600) AssertionError: Tensor-likes are not close! |
| diffusion_gemma | single | test_diffusion_gemma_chat_template | output_mismatch | 7/7 | (line 897) AssertionError: Lists differ: [[2, [46 chars]23613, 236761, 106, 107, 105, 4368, 107]] != [[2, [46 chars]23613, 236761, 106, 107, 105, 4368, 107, 100, 45518, 107, 101]] |
| diffusion_gemma | single | test_minified_diffusion_gemma_forward_batched | output_mismatch | 7/7 | (line 1694) AssertionError: Tensor-likes are not close! |
| diffusion_gemma | single | test_minified_diffusion_gemma_forward_text_only | output_mismatch | 7/7 | (line 1507) AssertionError: Tensor-likes are not close! |
| diffusion_gemma | single | test_minified_diffusion_gemma_forward_with_image | output_mismatch | 7/7 | (line 1600) AssertionError: Tensor-likes are not close! |
| efficientnet | multi | test_inference_image_classification_head | output_mismatch | 7/7 | (line 259) AssertionError: Tensor-likes are not close! |
| efficientnet | single | test_inference_image_classification_head | output_mismatch | 7/7 | (line 259) AssertionError: Tensor-likes are not close! |
| eomt_dinov3 | multi | test_inference_bf16 | output_mismatch | 7/7 | (line 315) AssertionError: Tensor-likes are not close! |
| eomt_dinov3 | single | test_inference_bf16 | output_mismatch | 7/7 | (line 315) AssertionError: Tensor-likes are not close! |
| exaone4 | multi | test_model_logits | output_mismatch | 7/7 | (line 88) AssertionError: Tensor-likes are not close! |
| exaone4 | single | test_model_generation_beyond_sliding_window | output_mismatch | 7/7 | (line 149) AssertionError: " Thi[46 chars] and the atmosphere is so relaxing. I'm gratef[47 chars]. It" != " Thi[46 chars] and I'm grateful for the opportunity to exper[26 chars]reak" |
| exaone4 | single | test_model_logits | output_mismatch | 7/7 | (line 88) AssertionError: Tensor-likes are not close! |
| exaone_moe | multi | test_model_logits | output_mismatch | 7/7 | (line 110) AssertionError: Tensor-likes are not close! |
| exaone_moe | single | test_model_logits | output_mismatch | 7/7 | (line 110) AssertionError: Tensor-likes are not close! |
| fastspeech2_conformer | multi | test_training_integration | output_mismatch | 7/7 | (line 453) AssertionError: Tensor-likes are not close! |
| fastspeech2_conformer | single | test_training_integration | output_mismatch | 7/7 | (line 453) AssertionError: Tensor-likes are not close! |
| fuyu | multi | test_greedy_generation | output_mismatch | 7/7 | (line 295) AssertionError: '\x04 A bus parked on the side of a road.' != 'A blue bus parked on the side of a road.' |
| fuyu | single | test_greedy_generation | output_mismatch | 7/7 | (line 295) AssertionError: '\x04 A bus parked on the side of a road.' != 'A blue bus parked on the side of a road.' |
| generation | multi | test_beam_search_early_stop_heuristic | output_mismatch | 7/7 | (line 3421) AssertionError: '<|user|>\nWhat is 3+5?\n<|assistant|>\nT[215 chars]d 8.' != "<|user|>\nWhat is 3+5?\n<|assistant|>\nT[282 chars]\\)." |
| generation | multi | test_mtp_use_correct_device_when_drafting | output_mismatch | 7/7 | (line 4095) AssertionError: device(type='cuda', index=0) != device(type='cuda', index=1) |
| generation | single | test_beam_search_early_stop_heuristic | output_mismatch | 7/7 | (line 3421) AssertionError: '<|user|>\nWhat is 3+5?\n<|assistant|>\nT[215 chars]d 8.' != "<|user|>\nWhat is 3+5?\n<|assistant|>\nT[282 chars]\\)." |
| granite | multi | test_model_3b_logits_bf16 | output_mismatch | 7/7 | (line 687) AssertionError: False is not true |
| granite | single | test_model_3b_logits_bf16 | output_mismatch | 7/7 | (line 687) AssertionError: False is not true |
| grounding_dino | multi | test_cross_attention_mask | output_mismatch | 7/7 | (line 787) AssertionError: Tensor-likes are not close! |
| grounding_dino | multi | test_grounding_dino_loss | output_mismatch | 7/7 | (line 869) AssertionError: Scalars are not close! |
| grounding_dino | multi | test_inference_object_detection_head | output_mismatch | 7/7 | (line 678) AssertionError: Tensor-likes are not close! |
| grounding_dino | multi | test_inference_object_detection_head_equivalence_cpu_accelerator | output_mismatch | 7/7 | (line 745) AssertionError: Tensor-likes are not close! |
| grounding_dino | single | test_cross_attention_mask | output_mismatch | 7/7 | (line 787) AssertionError: Tensor-likes are not close! |
| grounding_dino | single | test_grounding_dino_loss | output_mismatch | 7/7 | (line 869) AssertionError: Scalars are not close! |
| grounding_dino | single | test_inference_object_detection_head | output_mismatch | 7/7 | (line 678) AssertionError: Tensor-likes are not close! |
| grounding_dino | single | test_inference_object_detection_head_equivalence_cpu_accelerator | output_mismatch | 7/7 | (line 745) AssertionError: Tensor-likes are not close! |
| hrm_text | multi | test_forward_logits | output_mismatch | 7/7 | (line 280) AssertionError: Tensor-likes are not close! |
| hrm_text | single | test_forward_logits | output_mismatch | 7/7 | (line 280) AssertionError: Tensor-likes are not close! |
| hy_v3 | multi | test_small_model_logits_batched | output_mismatch | 7/7 | (line 105) AssertionError: Tensor-likes are not close! |
| hy_v3 | single | test_small_model_logits_batched | output_mismatch | 7/7 | (line 105) AssertionError: Tensor-likes are not close! |
| jais2 | multi | test_model_logits | output_mismatch | 7/7 | (line 118) AssertionError: Tensor-likes are not close! |
| jais2 | single | test_model_logits | output_mismatch | 7/7 | (line 118) AssertionError: Tensor-likes are not close! |
| jamba | multi | test_simple_batched_generate_with_padding | output_mismatch | 7/7 | (line 582) AssertionError: "<|startoftext|>Tell me a story<|pad|><|p[50 chars]t I'" != '<|pad|><|pad|><|pad|><|pad|><|pad|><|pad[76 chars]ates' |
| jamba | single | test_simple_batched_generate_with_padding | output_mismatch | 7/7 | (line 582) AssertionError: "<|startoftext|>Tell me a story<|pad|><|p[50 chars]t I'" != '<|pad|><|pad|><|pad|><|pad|><|pad|><|pad[76 chars]ates' |
| kimi_k25 | single | test_model_logits | output_mismatch | 7/7 | (line 324) AssertionError: Tensor-likes are not close! |
| kimi_k25 | single | test_model_logits_batched | output_mismatch | 7/7 | (line 389) AssertionError: Tensor-likes are not close! |
| kosmos2 | multi | test_inference_interpolate_pos_encoding | output_mismatch | 7/7 | (line 783) AssertionError: Tensor-likes are not close! |
| kosmos2 | multi | test_snowman_image_captioning | output_mismatch | 7/7 | (line 550) AssertionError: |
| kosmos2 | multi | test_snowman_image_captioning_batch | output_mismatch | 7/7 | (line 712) AssertionError: Lists differ: ['<gr[35 chars]ail: A snowman is sitting in front of a fire, [575 chars]t>.'] != ['<gr[35 chars]ail: The image features a snowman sitting by<p[836 chars]t>.'] |
| kosmos2 | single | test_inference_interpolate_pos_encoding | output_mismatch | 7/7 | (line 783) AssertionError: Tensor-likes are not close! |
| kosmos2 | single | test_snowman_image_captioning | output_mismatch | 7/7 | (line 550) AssertionError: |
| kosmos2 | single | test_snowman_image_captioning_batch | output_mismatch | 7/7 | (line 712) AssertionError: Lists differ: ['<gr[35 chars]ail: A snowman is sitting in front of a fire, [575 chars]t>.'] != ['<gr[35 chars]ail: The image features a snowman sitting by<p[836 chars]t>.'] |
| kosmos2_5 | multi | test_eager | output_mismatch | 7/7 | (line 585) AssertionError: Lists differ: ['<bb[65 chars]<y_612></bbox>[REG] BLACK SAKURA\n<bbox><x_690[603 chars]0\n'] != ['<bb[65 chars]<y_611></bbox>[REG] BLACK SAKURA\n<bbox><x_690[603 chars]0\n'] |
| kosmos2_5 | single | test_eager | output_mismatch | 7/7 | (line 585) AssertionError: Lists differ: ['<bb[65 chars]<y_612></bbox>[REG] BLACK SAKURA\n<bbox><x_690[603 chars]0\n'] != ['<bb[65 chars]<y_611></bbox>[REG] BLACK SAKURA\n<bbox><x_690[603 chars]0\n'] |
| layoutlmv2 | multi | test_processor_case_1 | output_mismatch | 7/7 | (line 213) AssertionError: Sequences differ: "[CLS[522 chars]t itc ' s new fmcg businesses are the fastest [829 chars]PAD]" != "[CLS[522 chars]t itc's new fmcg businesses are the fastest gr[827 chars]PAD]" |
| layoutlmv2 | multi | test_processor_case_4 | output_mismatch | 7/7 | (line 358) AssertionError: Sequences differ: "[CLS] what ' s his name? [SEP] 11 : 14 to 11 : 39 a[1108 chars]SEP]" != "[CLS] what's his name? [SEP] 11 : 14 to 11 : 39 a. [1106 chars]SEP]" |
| layoutlmv2 | multi | test_processor_case_5 | output_mismatch | 7/7 | (line 406) AssertionError: Sequences differ: "[CLS] what ' s his name? [SEP] hello world [SEP]" != "[CLS] what's his name? [SEP] hello world [SEP]" |
| layoutlmv2 | single | test_processor_case_1 | output_mismatch | 7/7 | (line 213) AssertionError: Sequences differ: "[CLS[522 chars]t itc ' s new fmcg businesses are the fastest [829 chars]PAD]" != "[CLS[522 chars]t itc's new fmcg businesses are the fastest gr[827 chars]PAD]" |
| layoutlmv2 | single | test_processor_case_4 | output_mismatch | 7/7 | (line 358) AssertionError: Sequences differ: "[CLS] what ' s his name? [SEP] 11 : 14 to 11 : 39 a[1108 chars]SEP]" != "[CLS] what's his name? [SEP] 11 : 14 to 11 : 39 a. [1106 chars]SEP]" |
| layoutlmv2 | single | test_processor_case_5 | output_mismatch | 7/7 | (line 406) AssertionError: Sequences differ: "[CLS] what ' s his name? [SEP] hello world [SEP]" != "[CLS] what's his name? [SEP] hello world [SEP]" |
| llava_next_video | multi | test_small_model_integration_test | output_mismatch | 7/7 | (line 387) AssertionError: 'USER[154 chars]hile wearing a pair of glasses that are too la[24 chars] are' != 'USER[154 chars]hile another child is attempting to read the s[45 chars]eems' |
| llava_next_video | single | test_small_model_integration_test | output_mismatch | 7/7 | (line 387) AssertionError: 'USER[154 chars]hile wearing a pair of glasses that are too la[24 chars] are' != 'USER[154 chars]hile another child is attempting to read the s[45 chars]eems' |
| luke | multi | test_inference_base_model | output_mismatch | 7/7 | (line 905) AssertionError: Tensor-likes are not close! |
| luke | multi | test_inference_large_model | output_mismatch | 7/7 | (line 940) AssertionError: Tensor-likes are not close! |
| luke | single | test_inference_base_model | output_mismatch | 7/7 | (line 905) AssertionError: Tensor-likes are not close! |
| luke | single | test_inference_large_model | output_mismatch | 7/7 | (line 940) AssertionError: Tensor-likes are not close! |
| lw_detr | multi | test_inference_object_detection_head_tiny | output_mismatch | 7/7 | (line 690) AssertionError: Tensor-likes are not close! |
| lw_detr | multi | test_inference_object_detection_head_xlarge | output_mismatch | 7/7 | (line 766) AssertionError: Tensor-likes are not close! |
| lw_detr | single | test_inference_object_detection_head_tiny | output_mismatch | 7/7 | (line 690) AssertionError: Tensor-likes are not close! |
| lw_detr | single | test_inference_object_detection_head_xlarge | output_mismatch | 7/7 | (line 766) AssertionError: Tensor-likes are not close! |
| m2m_100 | multi | test_seq_to_seq_generation | output_mismatch | 7/7 | (line 397) AssertionError: assert ['</s>__en__T... France.</s>'] == ['</s> __en__... France.</s>'] |
| m2m_100 | single | test_seq_to_seq_generation | output_mismatch | 7/7 | (line 397) AssertionError: assert ['</s>__en__T... France.</s>'] == ['</s> __en__... France.</s>'] |
| mimi | multi | test_integration | output_mismatch | 7/7 | (line 687) AssertionError: np.False_ is not true |
| mimi | multi | test_integration_longform | output_mismatch | 7/7 | (line 687) AssertionError: np.False_ is not true |
| mimi | single | test_integration | output_mismatch | 7/7 | (line 687) AssertionError: np.False_ is not true |
| mimi | single | test_integration_longform | output_mismatch | 7/7 | (line 687) AssertionError: np.False_ is not true |
| ministral | multi | test_model_8b_logits | output_mismatch | 7/7 | (line 93) AssertionError: Tensor-likes are not close! |
| ministral | single | test_model_8b_logits | output_mismatch | 7/7 | (line 93) AssertionError: Tensor-likes are not close! |
| ministral3 | multi | test_model_3b_generation | output_mismatch | 7/7 | (line 130) AssertionError: 'My favourite condiment is 100% pure oliv[64 chars] can' != "My favourite condiment is 100% pure oliv[46 chars]t in" |
| ministral3 | multi | test_model_3b_logits | output_mismatch | 7/7 | (line 102) AssertionError: Tensor-likes are not close! |
| ministral3 | single | test_model_3b_generation | output_mismatch | 7/7 | (line 130) AssertionError: 'My favourite condiment is 100% pure oliv[64 chars] can' != "My favourite condiment is 100% pure oliv[46 chars]t in" |
| ministral3 | single | test_model_3b_logits | output_mismatch | 7/7 | (line 102) AssertionError: Tensor-likes are not close! |
| mixtral | multi | test_small_model_logits | output_mismatch | 7/7 | (line 143) AssertionError: Tensor-likes are not close! |
| mixtral | multi | test_small_model_logits_batched | output_mismatch | 7/7 | (line 188) AssertionError: Tensor-likes are not close! |
| mixtral | single | test_small_model_logits | output_mismatch | 7/7 | (line 143) AssertionError: Tensor-likes are not close! |
| mixtral | single | test_small_model_logits_batched | output_mismatch | 7/7 | (line 188) AssertionError: Tensor-likes are not close! |
| mllama | multi | test_11b_model_integration_batched_generate | output_mismatch | 7/7 | (line 699) AssertionError: "This image showst's a photo of a person named I'm not abl[59 chars]ge's" != "This image shows\nI'm not able to provide information on [64 chars]ning" |
| mllama | multi | test_11b_model_integration_multi_image_generate | output_mismatch | 7/7 | (line 762) AssertionError: 'The image shows a red octagonal stop sign w[59 chars]to a' != 'This image shows a long wooden dock extendi[67 chars]ling' |
| mllama | single | test_11b_model_integration_batched_generate | output_mismatch | 7/7 | (line 699) AssertionError: "This image showst's a photo of a person named I'm not abl[59 chars]ge's" != "This image shows\nI'm not able to provide information on [64 chars]ning" |
| mllama | single | test_11b_model_integration_multi_image_generate | output_mismatch | 7/7 | (line 762) AssertionError: 'The image shows a red octagonal stop sign w[59 chars]to a' != 'This image shows a long wooden dock extendi[67 chars]ling' |
| mluke | multi | test_entity_classification_no_padding_or_truncation | output_mismatch | 7/7 | (line 453) AssertionError: '<s> Japanese is an<s> East Asian language<s> spoken by about[40 chars]</s>' != '<s> Japanese is an<ent>East Asian language<ent>spoken by abo[42 chars]</s>' |
| mluke | multi | test_entity_pair_classification_no_padding_or_truncation | output_mismatch | 7/7 | (line 507) AssertionError: '<s><s> Japanese<s> is an East Asian language [64 chars]</s>' != '<s><ent>Japanese<ent>is an East Asian languag[68 chars]</s>' |
| mluke | multi | test_entity_span_classification_no_padding_or_truncation | output_mismatch | 7/7 | (line 572) AssertionError: '<s> [33 chars]e spoken by about 128 million people, primarily in Japan .</s>' != '<s> [33 chars]e spoken by about 128 million people, primarily in Japan.</s>' |
| mluke | single | test_entity_classification_no_padding_or_truncation | output_mismatch | 7/7 | (line 453) AssertionError: '<s> Japanese is an<s> East Asian language<s> spoken by about[40 chars]</s>' != '<s> Japanese is an<ent>East Asian language<ent>spoken by abo[42 chars]</s>' |
| mluke | single | test_entity_pair_classification_no_padding_or_truncation | output_mismatch | 7/7 | (line 507) AssertionError: '<s><s> Japanese<s> is an East Asian language [64 chars]</s>' != '<s><ent>Japanese<ent>is an East Asian languag[68 chars]</s>' |
| mluke | single | test_entity_span_classification_no_padding_or_truncation | output_mismatch | 7/7 | (line 572) AssertionError: '<s> [33 chars]e spoken by about 128 million people, primarily in Japan .</s>' != '<s> [33 chars]e spoken by about 128 million people, primarily in Japan.</s>' |
| mm_grounding_dino | multi | test_inference_object_detection_head | output_mismatch | 7/7 | (line 672) AssertionError: Tensor-likes are not close! |
| mm_grounding_dino | multi | test_inference_object_detection_head_equivalence_cpu_gpu | output_mismatch | 7/7 | (line 738) AssertionError: Tensor-likes are not close! |
| mm_grounding_dino | multi | test_mm_grounding_dino_loss | output_mismatch | 7/7 | (line 687) AssertionError: False is not true |
| mm_grounding_dino | single | test_inference_object_detection_head | output_mismatch | 7/7 | (line 672) AssertionError: Tensor-likes are not close! |
| mm_grounding_dino | single | test_inference_object_detection_head_equivalence_cpu_gpu | output_mismatch | 7/7 | (line 738) AssertionError: Tensor-likes are not close! |
| mm_grounding_dino | single | test_mm_grounding_dino_loss | output_mismatch | 7/7 | (line 687) AssertionError: False is not true |
| moshi | multi | test_moshika_greedy_unconditional_fp16 | output_mismatch | 7/7 | (line 687) AssertionError: np.False_ is not true |
| moshi | multi | test_moshiko_greedy_unconditional_fp16 | output_mismatch | 7/7 | (line 687) AssertionError: np.False_ is not true |
| moshi | multi | test_moshiko_greedy_unconditional_fp16_eager | output_mismatch | 7/7 | (line 687) AssertionError: False is not true |
| moshi | multi | test_moshiko_greedy_unconditional_fp32 | output_mismatch | 7/7 | (line 687) AssertionError: np.False_ is not true |
| moshi | single | test_moshika_greedy_unconditional_fp16 | output_mismatch | 7/7 | (line 687) AssertionError: np.False_ is not true |
| moshi | single | test_moshiko_greedy_unconditional_fp16 | output_mismatch | 7/7 | (line 687) AssertionError: np.False_ is not true |
| moshi | single | test_moshiko_greedy_unconditional_fp16_eager | output_mismatch | 7/7 | (line 687) AssertionError: False is not true |
| moshi | single | test_moshiko_greedy_unconditional_fp32 | output_mismatch | 7/7 | (line 687) AssertionError: np.False_ is not true |
| musicgen | multi | test_generate_text_prompt_sampling | output_mismatch | 7/7 | (line 1263) AssertionError: Tensor-likes are not close! |
| musicgen | multi | test_generate_unconditional_sampling | output_mismatch | 7/7 | (line 1180) AssertionError: Tensor-likes are not close! |
| musicgen | single | test_generate_text_prompt_sampling | output_mismatch | 7/7 | (line 1263) AssertionError: Tensor-likes are not close! |
| musicgen | single | test_generate_unconditional_sampling | output_mismatch | 7/7 | (line 1180) AssertionError: Tensor-likes are not close! |
| nemotron | multi | test_nemotron_8b_generation_eager | output_mismatch | 7/7 | (line 106) AssertionError: Lists differ: ['Wha[46 chars]er: Jupiter\n\nWhat is the answer'] != ['Wha[46 chars]er: Jupiter\n\nWhat is the answer: What is the name of the 19'] |
| nemotron | single | test_nemotron_8b_generation_eager | output_mismatch | 7/7 | (line 106) AssertionError: Lists differ: ['Wha[46 chars]er: Jupiter\n\nWhat is the answer'] != ['Wha[46 chars]er: Jupiter\n\nWhat is the answer: What is the name of the 19'] |
| nllb_moe | multi | test_inference_logits | output_mismatch | 7/7 | (line 399) AssertionError: Tensor-likes are not close! |
| nllb_moe | single | test_inference_logits | output_mismatch | 7/7 | (line 399) AssertionError: Tensor-likes are not close! |
| oneformer | multi | test_inference_no_head | output_mismatch | 7/7 | (line 498) AssertionError: Tensor-likes are not close! |
| oneformer | multi | test_inference_universal_segmentation_head | output_mismatch | 7/7 | (line 540) AssertionError: Tensor-likes are not close! |
| oneformer | single | test_inference_no_head | output_mismatch | 7/7 | (line 498) AssertionError: Tensor-likes are not close! |
| oneformer | single | test_inference_universal_segmentation_head | output_mismatch | 7/7 | (line 540) AssertionError: Tensor-likes are not close! |
| ovis2 | multi | test_small_model_integration_test_batch_different_resolutions | output_mismatch | 7/7 | (line 354) AssertionError: Lists differ: ['sys[81 chars]ant\n', 'system\nYou are a helpful assistant.\[139 chars]et.'] != ['sys[81 chars]ant\nAnswer: I see a brown dog standing on a w[224 chars]et.'] |
| ovis2 | single | test_small_model_integration_test_batch_different_resolutions | output_mismatch | 7/7 | (line 354) AssertionError: Lists differ: ['sys[81 chars]ant\n', 'system\nYou are a helpful assistant.\[139 chars]et.'] != ['sys[81 chars]ant\nAnswer: I see a brown dog standing on a w[224 chars]et.'] |
| persimmon | multi | test_model_8b_chat_logits | output_mismatch | 7/7 | (line 99) AssertionError: Tensor-likes are not close! |
| persimmon | single | test_model_8b_chat_logits | output_mismatch | 7/7 | (line 99) AssertionError: Tensor-likes are not close! |
| plbart | multi | test_fill_mask | output_mismatch | 7/7 | (line 444) AssertionError: '0 0 the 0 the 0 the 0 the 0 the 0 the 0 the 0 the 0' != '0 0 the 0 the 0 the 0 the 0 the 0 the 0 the 0 the' |
| plbart | multi | test_java_cs_generate_batch | output_mismatch | 7/7 | (line 379) AssertionError: assert ['public int ...turn a * b *'] == ['public int ...rn a * b * c'] |
| plbart | multi | test_java_cs_generate_one | output_mismatch | 7/7 | (line 370) AssertionError: 'public int maximum(int a, int b, int c){return Math.Max(' != 'public int maximum(int a, int b, int c){return Math.Max(a' |
| plbart | single | test_fill_mask | output_mismatch | 7/7 | (line 444) AssertionError: '0 0 the 0 the 0 the 0 the 0 the 0 the 0 the 0 the 0' != '0 0 the 0 the 0 the 0 the 0 the 0 the 0 the 0 the' |
| plbart | single | test_java_cs_generate_batch | output_mismatch | 7/7 | (line 379) AssertionError: assert ['public int ...turn a * b *'] == ['public int ...rn a * b * c'] |
| plbart | single | test_java_cs_generate_one | output_mismatch | 7/7 | (line 370) AssertionError: 'public int maximum(int a, int b, int c){return Math.Max(' != 'public int maximum(int a, int b, int c){return Math.Max(a' |
| pvt | multi | test_inference_image_classification | output_mismatch | 7/7 | (line 257) AssertionError: Tensor-likes are not close! |
| pvt | multi | test_inference_model | output_mismatch | 7/7 | (line 284) AssertionError: Tensor-likes are not close! |
| pvt | single | test_inference_image_classification | output_mismatch | 7/7 | (line 257) AssertionError: Tensor-likes are not close! |
| pvt | single | test_inference_model | output_mismatch | 7/7 | (line 284) AssertionError: Tensor-likes are not close! |
| pvt_v2 | multi | test_inference_image_classification | output_mismatch | 7/7 | (line 275) AssertionError: Tensor-likes are not close! |
| pvt_v2 | multi | test_inference_model | output_mismatch | 7/7 | (line 292) AssertionError: torch.Size([1, 256, 7, 7]) != torch.Size([1, 50, 512]) |
| pvt_v2 | single | test_inference_image_classification | output_mismatch | 7/7 | (line 275) AssertionError: Tensor-likes are not close! |
| pvt_v2 | single | test_inference_model | output_mismatch | 7/7 | (line 292) AssertionError: torch.Size([1, 256, 7, 7]) != torch.Size([1, 50, 512]) |
| qwen2_moe | multi | test_model_a2_7b_logits | output_mismatch | 7/7 | (line 157) AssertionError: Tensor-likes are not close! |
| qwen2_moe | single | test_model_a2_7b_logits | output_mismatch | 7/7 | (line 157) AssertionError: Tensor-likes are not close! |
| qwen3 | multi | test_model_600m_logits | output_mismatch | 7/7 | (line 92) AssertionError: Tensor-likes are not close! |
| qwen3 | multi | test_speculative_generation | output_mismatch | 7/7 | (line 198) AssertionError: 'My f[22 chars]100% beef, 100% beef, 100% beef.' != 'My f[22 chars]100% vegetable oil. It has a rich, creamy text[19 chars]utty' |
| qwen3 | single | test_model_600m_logits | output_mismatch | 7/7 | (line 92) AssertionError: Tensor-likes are not close! |
| qwen3 | single | test_speculative_generation | output_mismatch | 7/7 | (line 198) AssertionError: 'My f[22 chars]100% beef, 100% beef, 100% beef.' != 'My f[22 chars]100% vegetable oil. It has a rich, creamy text[19 chars]utty' |
| qwen3_5 | multi | test_model_video_generation | output_mismatch | 7/7 | (line 852) AssertionError: Lists differ: [248045, 846, 198, 27, 15, 13, 18, 6283, 29, 248053] != [248045, 846, 198, 248053, 27, 15, 13, 18, 6283, 29] |
| qwen3_5 | multi | test_model_video_generation_batch | output_mismatch | 7/7 | (line 906) AssertionError: Lists differ: [248045, 846, 198, 27, 15, 13, 18, 6283, 29, 248053] != [248045, 846, 198, 248053, 27, 15, 13, 18, 6283, 29] |
| qwen3_5 | single | test_model_video_generation | output_mismatch | 7/7 | (line 852) AssertionError: Lists differ: [248045, 846, 198, 27, 15, 13, 18, 6283, 29, 248053] != [248045, 846, 198, 248053, 27, 15, 13, 18, 6283, 29] |
| qwen3_5 | single | test_model_video_generation_batch | output_mismatch | 7/7 | (line 906) AssertionError: Lists differ: [248045, 846, 198, 27, 15, 13, 18, 6283, 29, 248053] != [248045, 846, 198, 248053, 27, 15, 13, 18, 6283, 29] |
| rag | single | test_rag_sequence_generate_batch | output_mismatch | 7/7 | (line 948) AssertionError: Lists differ: [' michael gross', ' monday 17 , 2018', ' te[96 chars]ndo'] != [' albert einstein', ' june 22 , 2018', ' am[85 chars]' 8'] |
| rag | single | test_rag_sequence_generate_batch_from_context_input_ids | output_mismatch | 7/7 | (line 1000) AssertionError: Lists differ: [' michael gross', ' monday 17 , 2018', ' te[96 chars]ndo'] != [' albert einstein', ' june 22 , 2018', ' am[85 chars]' 8'] |
| rag | single | test_rag_sequence_generate_beam | output_mismatch | 7/7 | (line 892) AssertionError: '" in the United States. "People Need Love"[155 chars]hit.' != '"She\'s My Kind of Girl" was released thro[257 chars]nts.' |
| rag | single | test_rag_token_generate_beam | output_mismatch | 7/7 | (line 854) AssertionError: '"She[14 chars] Girl' != '"She[14 chars] Girl" was released through Epic Records in Ja[179 chars]ses"' |
| recurrent_gemma | multi | test_2b_generate | output_mismatch | 7/7 | (line 254) AssertionError: Lists differ: ['Hel[325 chars]oday the 1990s, the 1990s, the 1990s, the 1990[43 chars]0s,'] != ['Hel[325 chars]oday is a new app that allows you to make mone[256 chars]app'] |
| recurrent_gemma | multi | test_2b_sample | output_mismatch | 7/7 | (line 292) AssertionError: Lists differ: ['Wha[24 chars]Deep Learning (or deep learning) is one of the[107 chars]ple'] != ['Wha[24 chars]Deep learning is the next frontier in computer[98 chars] is'] |
| recurrent_gemma | multi | test_longer_than_window | output_mismatch | 7/7 | (line 340) AssertionError: Lists differ: [' Jean-Philippe Guillet said, "We have no[245 chars]eo.'] != [" Robin's comments follow claims by two m[249 chars]the"] |
| recurrent_gemma | single | test_2b_generate | output_mismatch | 7/7 | (line 254) AssertionError: Lists differ: ['Hel[325 chars]oday the 1990s, the 1990s, the 1990s, the 1990[43 chars]0s,'] != ['Hel[325 chars]oday is a new app that allows you to make mone[256 chars]app'] |
| recurrent_gemma | single | test_2b_sample | output_mismatch | 7/7 | (line 292) AssertionError: Lists differ: ['Wha[24 chars]Deep Learning (or deep learning) is one of the[107 chars]ple'] != ['Wha[24 chars]Deep learning is the next frontier in computer[98 chars] is'] |
| recurrent_gemma | single | test_longer_than_window | output_mismatch | 7/7 | (line 340) AssertionError: Lists differ: [' Jean-Philippe Guillet said, "We have no[245 chars]eo.'] != [" Robin's comments follow claims by two m[249 chars]the"] |
| reformer | multi | test_pretrained_generate_crime_and_punish | output_mismatch | 7/7 | (line 1357) AssertionError: 'A fe[36 chars]is ideas, so attentively two or three thousand roubles, and' != 'A fe[36 chars]is ideas, at the first entrance. He was positively for an inst' |
| reformer | single | test_pretrained_generate_crime_and_punish | output_mismatch | 7/7 | (line 1357) AssertionError: 'A fe[36 chars]is ideas, so attentively two or three thousand roubles, and' != 'A fe[36 chars]is ideas, at the first entrance. He was positively for an inst' |
| regnet | multi | test_inference_image_classification_head | output_mismatch | 7/7 | (line 243) AssertionError: Tensor-likes are not close! |
| regnet | single | test_inference_image_classification_head | output_mismatch | 7/7 | (line 243) AssertionError: Tensor-likes are not close! |
| seamless_m4t_v2 | multi | test_to_rus_speech | output_mismatch | 7/7 | (line 1061) AssertionError: Lists differ: [3, 256074, 107, 248213, 404, 247792, 247789, 3] != [[3, 256074, 107, 248213, 404, 247792, 247[53 chars], 3]] |
| seamless_m4t_v2 | single | test_to_rus_speech | output_mismatch | 7/7 | (line 1061) AssertionError: Lists differ: [3, 256074, 107, 248213, 404, 247792, 247789, 3] != [[3, 256074, 107, 248213, 404, 247792, 247[53 chars], 3]] |
| smollm3 | multi | test_export_static_cache | output_mismatch | 7/7 | (line 198) AssertionError: 'Gravity is the force that pulls objects [69 chars] and' != ["Gravity is the force that pulls objects[85 chars] of"] |
| smollm3 | multi | test_model_3b_logits | output_mismatch | 7/7 | (line 89) AssertionError: Tensor-likes are not close! |
| smollm3 | single | test_export_static_cache | output_mismatch | 7/7 | (line 198) AssertionError: 'Gravity is the force that pulls objects [69 chars] and' != ["Gravity is the force that pulls objects[85 chars] of"] |
| smollm3 | single | test_model_3b_logits | output_mismatch | 7/7 | (line 89) AssertionError: Tensor-likes are not close! |
| starcoder2 | multi | test_starcoder2_batched_generation_4bit | output_mismatch | 7/7 | (line 152) AssertionError: Lists differ: ['Hel[207 chars]ld():\n\treturn "Hello World"\n\n@app.route(\'[76 chars]ute'] != ['Hel[207 chars]ld(): hello_world():\n return "Hello World![89 chars]n\n'] |
| starcoder2 | multi | test_starcoder2_batched_generation_eager | output_mismatch | 7/7 | (line 99) AssertionError: Lists differ: ['Hel[223 chars]ld():\n\treturn 'Hello World!'\n\n@app.route('[72 chars]app"] != ['Hel[223 chars]ld(): hello_world():\n return 'Hello World![87 chars]n\n"] |
| starcoder2 | multi | test_starcoder2_batched_generation_sdpa | output_mismatch | 7/7 | (line 79) AssertionError: Lists differ: ['Hel[223 chars]ld():\n\treturn 'Hello World!'\n\n@app.route('[72 chars]app"] != ['Hel[223 chars]ld(): hello_world():\n return 'Hello World![87 chars]n\n"] |
| starcoder2 | single | test_starcoder2_batched_generation_4bit | output_mismatch | 7/7 | (line 152) AssertionError: Lists differ: ['Hel[207 chars]ld():\n\treturn "Hello World"\n\n@app.route(\'[76 chars]ute'] != ['Hel[207 chars]ld(): hello_world():\n return "Hello World![89 chars]n\n'] |
| starcoder2 | single | test_starcoder2_batched_generation_eager | output_mismatch | 7/7 | (line 99) AssertionError: Lists differ: ['Hel[223 chars]ld():\n\treturn 'Hello World!'\n\n@app.route('[72 chars]app"] != ['Hel[223 chars]ld(): hello_world():\n return 'Hello World![87 chars]n\n"] |
| starcoder2 | single | test_starcoder2_batched_generation_sdpa | output_mismatch | 7/7 | (line 79) AssertionError: Lists differ: ['Hel[223 chars]ld():\n\treturn 'Hello World!'\n\n@app.route('[72 chars]app"] != ['Hel[223 chars]ld(): hello_world():\n return 'Hello World![87 chars]n\n"] |
| superpoint | multi | test_inference | output_mismatch | 7/7 | (line 276) AssertionError: torch.Size([2, 786, 2]) != torch.Size([2, 830, 2]) |
| superpoint | single | test_inference | output_mismatch | 7/7 | (line 276) AssertionError: torch.Size([2, 786, 2]) != torch.Size([2, 830, 2]) |
| swinv2 | multi | test_inference_fp16 | output_mismatch | 7/7 | (line 487) AssertionError: Tensor-likes are not close! |
| swinv2 | single | test_inference_fp16 | output_mismatch | 7/7 | (line 487) AssertionError: Tensor-likes are not close! |
| t5 | multi | test_compile_static_cache | output_mismatch | 7/7 | (line 1444) AssertionError: Lists differ: ['the[91 chars]rames . the laws of physics are the same for a[65 chars]t .'] != ['the[91 chars]rames. the laws of physics are the same for al[62 chars]nt.'] |
| t5 | single | test_compile_static_cache | output_mismatch | 7/7 | (line 1444) AssertionError: Lists differ: ['the[91 chars]rames . the laws of physics are the same for a[65 chars]t .'] != ['the[91 chars]rames. the laws of physics are the same for al[62 chars]nt.'] |
| univnet | multi | test_integration | output_mismatch | 7/7 | (line 326) AssertionError: Scalars are not close! |
| univnet | single | test_integration | output_mismatch | 7/7 | (line 326) AssertionError: Scalars are not close! |
| utils | multi | test_cache_copy | output_mismatch | 7/7 | (line 670) AssertionError: Lists differ: ['You are a helpful assistant. Help me to [390 chars] is'] != ["You are a helpful assistant. Help me to [385 chars] is'] |
| utils | multi | test_dynamic_cache_hard | output_mismatch | 7/7 | (line 522) AssertionError: "Here[57 chars]ave fur, they have four legs, they have a tail[1045 chars]have" != "Here[57 chars]ave four legs, they have a tail, they have a f[1078 chars]They" |
| utils | single | test_cache_copy | output_mismatch | 7/7 | (line 670) AssertionError: Lists differ: ['You are a helpful assistant. Help me to [390 chars] is'] != ["You are a helpful assistant. Help me to [385 chars] is'] |
| utils | single | test_dynamic_cache_hard | output_mismatch | 7/7 | (line 522) AssertionError: "Here[57 chars]ave fur, they have four legs, they have a tail[1045 chars]have" != "Here[57 chars]ave four legs, they have a tail, they have a f[1078 chars]They" |
| video_llava | multi | test_small_model_integration_test_llama | output_mismatch | 7/7 | (line 491) AssertionError: 'USER: \nDescribe the video in details. A[572 chars]ion.' != "USER: \nDescribe the video in details. A[675 chars]ing." |
| video_llava | single | test_small_model_integration_test_llama | output_mismatch | 7/7 | (line 491) AssertionError: 'USER: \nDescribe the video in details. A[572 chars]ion.' != "USER: \nDescribe the video in details. A[675 chars]ing." |
| videomae | multi | test_inference_for_pretraining | output_mismatch | 7/7 | (line 482) AssertionError: The values for attribute 'shape' do not match: torch.Size([]) != torch.Size([1]). |
| videomae | single | test_inference_for_pretraining | output_mismatch | 7/7 | (line 482) AssertionError: The values for attribute 'shape' do not match: torch.Size([]) != torch.Size([1]). |
| vilt | multi | test_inference_masked_lm | output_mismatch | 7/7 | (line 573) AssertionError: Tensor-likes are not close! |
| vilt | single | test_inference_masked_lm | output_mismatch | 7/7 | (line 573) AssertionError: Tensor-likes are not close! |
| vits | multi | test_forward_fp16 | output_mismatch | 7/7 | (line 409) AssertionError: Tensor-likes are not close! |
| vits | single | test_forward_fp16 | output_mismatch | 7/7 | (line 409) AssertionError: Tensor-likes are not close! |
| zamba | multi | test_simple_generate | output_mismatch | 7/7 | (line 475) AssertionError: The values for attribute 'dtype' do not match: torch.bfloat16 != torch.float32. |