INFO 10-05 23:03:35 [__init__.py:44] Available plugins for group vllm.platform_plugins: INFO 10-05 23:03:35 [__init__.py:46] - tt -> vllm_tt_plugin.entrypoints:platform_plugin INFO 10-05 23:03:35 [__init__.py:49] All plugins in this group will be loaded. Set `VLLM_PLUGINS` to control which plugins to load. INFO 10-05 23:03:36 [__init__.py:212] Platform plugin tt is activated INFO 10-05 23:03:39 [importing.py:44] Triton is installed but 0 active driver(s) found (expected 1). Disabling Triton to prevent runtime errors. INFO 10-05 23:03:39 [importing.py:68] Triton not installed or not compatible; certain GPU-related functions will not be available. (APIServer pid=1) INFO 10-05 23:03:40 [utils.py:287] (APIServer pid=1) INFO 10-05 23:03:40 [utils.py:287] █ █ █▄ ▄█ (APIServer pid=1) INFO 10-05 23:03:40 [utils.py:287] ▄▄ ▄█ █ █ █ ▀▄▀ █ version 0.1.dev14187+g03fa3af2e (APIServer pid=1) INFO 10-05 23:03:40 [utils.py:287] █▄█▀ █ █ █ █ model /model (APIServer pid=1) INFO 10-05 23:03:40 [utils.py:287] ▀▀ ▀▀▀▀▀ ▀▀▀▀▀ ▀ ▀ (APIServer pid=1) INFO 10-05 23:03:40 [utils.py:287] (APIServer pid=1) INFO 10-05 23:03:40 [utils.py:223] non-default args: {'host': '0.0.0.0', 'enable_auto_tool_choice': True, 'tool_call_parser': 'gemma4', 'model': '/model', 'max_model_len': 262144, 'served_model_name': ['google/gemma-4-12B-it'], 'reasoning_parser': 'gemma4', 'block_size': 64, 'enable_prefix_caching': True, 'max_num_batched_tokens': 16384, 'max_num_seqs': 1, 'enable_chunked_prefill': True, 'speculative_config': {'method': 'gemma4_assistant', 'model': '/drafter', 'num_speculative_tokens': 5}, 'additional_config': {'tt': {'trace_mode': 'all', 'enable_model_warmup': True, 'trace_region_size': 200000000}}} (APIServer pid=1) INFO 10-05 23:03:46 [model.py:526] Resolved architecture: Gemma4UnifiedForConditionalGeneration (APIServer pid=1) INFO 10-05 23:03:46 [model.py:1550] Using max model len 262144 (APIServer pid=1) INFO 10-05 23:03:46 [scheduler.py:224] Chunked prefill is enabled with max_num_batched_tokens=16384. (APIServer pid=1) WARNING 10-05 23:03:46 [vllm.py:662] Async scheduling not supported with gemma4_assistant-based speculative decoding and will be disabled. (APIServer pid=1) INFO 10-05 23:03:46 [vllm.py:698] Asynchronous scheduling is disabled. (APIServer pid=1) WARNING 10-05 23:03:46 [vllm.py:743] Inductor compilation was disabled by user settings, optimizations settings that are only active during inductor compilation will be ignored. (APIServer pid=1) INFO 10-05 23:03:47 [platform.py:796] Automatic prefix caching is enabled INFO 10-05 23:03:50 [__init__.py:44] Available plugins for group vllm.platform_plugins: INFO 10-05 23:03:50 [__init__.py:46] - tt -> vllm_tt_plugin.entrypoints:platform_plugin INFO 10-05 23:03:50 [__init__.py:49] All plugins in this group will be loaded. Set `VLLM_PLUGINS` to control which plugins to load. INFO 10-05 23:03:51 [__init__.py:212] Platform plugin tt is activated INFO 10-05 23:03:53 [importing.py:44] Triton is installed but 0 active driver(s) found (expected 1). Disabling Triton to prevent runtime errors. INFO 10-05 23:03:53 [importing.py:68] Triton not installed or not compatible; certain GPU-related functions will not be available. (EngineCore_DP0 pid=150) INFO 10-05 23:03:54 [core.py:98] Initializing a V1 LLM engine (v0.1.dev14187+g03fa3af2e) with config: model='/model', speculative_config=SpeculativeConfig(method='gemma4_assistant', model='/drafter', num_spec_tokens=5), tokenizer='/model', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=262144, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=True, quantization=None, enforce_eager=False, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=None, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='gemma4', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False), seed=0, served_model_name=google/gemma-4-12B-it, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': , 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'eager', 'custom_ops': ['all'], 'splitting_ops': ['vllm::unified_attention', 'vllm::unified_attention_with_output', 'vllm::unified_mla_attention', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::linear_attention', 'vllm::plamo2_mamba_mixer', 'vllm::gdn_attention_core', 'vllm::kda_attention', 'vllm::sparse_attn_indexer', 'vllm::rocm_aiter_sparse_attn_indexer', 'vllm::unified_kv_cache_update'], 'compile_mm_encoder': False, 'compile_sizes': None, 'compile_ranges_split_points': [16384], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': , 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': None, 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': True, 'fuse_act_quant': True, 'fuse_attn_quant': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False, 'fuse_act_padding': False}, 'max_cudagraph_capture_size': None, 'dynamic_shapes_config': {'type': , 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': False, 'static_all_moe_layers': []} (EngineCore_DP0 pid=150) INFO 10-05 23:03:54 [platform.py:796] Automatic prefix caching is enabled 2026-10-05 23:03:54.651 | info | Device | Opening user mode device driver (tt_cluster.cpp:228) 2026-10-05 23:03:54.651 | info | UMD | Cluster constructor started. (cluster.cpp:378) 2026-10-05 23:03:54.651 | info | UMD | Creating TopologyDiscovery for architecture: blackhole (topology_discovery.cpp:95) 2026-10-05 23:03:54.651 | info | UMD | Starting topology discovery. (topology_discovery.cpp:114) 2026-10-05 23:03:54.654 | info | UMD | Established firmware bundle version: 19.15.0 (topology_discovery.cpp:574) 2026-10-05 23:03:54.654 | info | UMD | Completed topology discovery. (topology_discovery.cpp:118) 2026-10-05 23:03:54.680 | warning | UMD | hwloc_set_area_membind(): failed for physical_device_id: 0 on NodeSet: [0] with errno: Operation not permitted (pid: 150 tid: 135962864665024) (cpuset_lib.cpp:440) 2026-10-05 23:03:54.680 | warning | UMD | ---- ttSiliconDevice::init_hugepage: bind_area_to_memory_nodeset() failed (physical_device_id: 0 ch: 0). Hugepage allocation is not on NumaNode matching TT Device. Side-Effect is decreased Device->Host perf (Issue #893). (silicon_sysmem_manager.cpp:271) 2026-10-05 23:03:54.680 | info | UMD | Opening local chip ids/PCIe ids: {0}/[0] and remote chip ids {} (cluster.cpp:168) 2026-10-05 23:03:54.680 | info | UMD | IOMMU: disabled (cluster.cpp:142) 2026-10-05 23:03:54.680 | info | UMD | KMD version: 2.11.0 (cluster.cpp:145) 2026-10-05 23:03:54.680 | info | UMD | Cluster constructor completed. (cluster.cpp:574) 2026-10-05 23:03:54.681 | info | UMD | Starting devices in cluster (cluster.cpp:1092) 2026-10-05 23:03:54.727 | info | UMD | Starting devices in cluster completed. (cluster.cpp:1100) (EngineCore_DP0 pid=150) INFO 10-05 23:03:54 [worker.py:781] Attempting to open mesh device with grid shape (1, 1) 2026-10-05 23:03:54.766 | info | Distributed | Using auto discovery to generate mesh graph. (metal_env.cpp:437) 2026-10-05 23:03:54.766 | info | Distributed | Constructing control plane using auto-discovery (no mesh graph descriptor). (metal_env.cpp:499) 2026-10-05 23:03:54.766 | warning | Always | Unknown motherboard 'ROG STRIX B650-A GAMING WIFI' for chip_id=0 (bus_id=0x1) — falling back to bus_id as tray_id. Add this motherboard and its bus IDs to mobo_to_bus_ids in physical_system_discovery.cpp. (physical_system_discovery.cpp:112) 2026-10-05 23:03:54.768 | info | Fabric | Logical multi-mesh adjacency: intermesh degree histogram {0:1}; intra-mesh degree histograms mesh0 {0:1} (topology_mapper_utils.cpp:2404) 2026-10-05 23:03:54.768 | info | Fabric | Physical multi-mesh adjacency: intermesh degree histogram {0:1}; intra-mesh degree histograms mesh0 {0:1} (topology_mapper_utils.cpp:2412) 2026-10-05 23:03:54.779 | info | BuildKernels | Using pre-compiled firmware from: /home/container_app_user/tt-metal/tt_metal/pre-compiled/15570547042316605318/ (build_env_manager.cpp:347) (EngineCore_DP0 pid=150) INFO 10-05 23:03:54 [worker.py:794] multidevice with 1 devices and grid (1, 1) is created (EngineCore_DP0 pid=150) INFO 10-05 23:03:54 [model_runner.py:182] TTModelRunner: trace_mode=all, sample_on_device_mode=None, enable_model_warmup=True (EngineCore_DP0 pid=150) WARNING 10-05 23:03:54 [__init__.py:204] min_p, logit_bias, and min_tokens parameters won't currently work with speculative decoding enabled. (EngineCore_DP0 pid=150) 2026-10-05 23:04:08.647 | INFO | models.demos.gemma4.tt.model_config:load_state_dict:143 - Loading 1 safetensor files from /drafter (EngineCore_DP0 pid=150) Loading safetensors: 0%| | 0/1 [00:00'. (EngineCore_DP0 pid=150) INFO 10-05 23:04:08 [kv_cache_utils.py:820] Overriding num_gpu_blocks=837723144972 with num_gpu_blocks_override=4097 (EngineCore_DP0 pid=150) INFO 10-05 23:04:08 [kv_cache_utils.py:1308] GPU KV cache size: 262,208 tokens (EngineCore_DP0 pid=150) INFO 10-05 23:04:08 [kv_cache_utils.py:1313] Maximum concurrency for 262,144 tokens per request: 1.00x 2026-10-05 23:04:08.731 | info | Metal | Finalize dfb: dfb_offset == base_offset: 80, dfb size: 32, return value: 112 (dataflow_buffer.cpp:1656) 2026-10-05 23:04:08.731 | info | Metal | Finalize dfb: dfb_offset == base_offset: 0, dfb size: 0, return value: 0 (dataflow_buffer.cpp:1656) 2026-10-05 23:04:08.731 | info | Metal | Finalize dfb: dfb_offset == base_offset: 0, dfb size: 0, return value: 0 (dataflow_buffer.cpp:1656) 2026-10-05 23:04:08.731 | info | Metal | Finalize dfb: dfb_offset == base_offset: 0, dfb size: 0, return value: 0 (dataflow_buffer.cpp:1656) 2026-10-05 23:04:09.032 | info | Metal | Finalize dfb: dfb_offset == base_offset: 80, dfb size: 32, return value: 112 (dataflow_buffer.cpp:1656) 2026-10-05 23:04:09.032 | info | Metal | Finalize dfb: dfb_offset == base_offset: 0, dfb size: 0, return value: 0 (dataflow_buffer.cpp:1656) 2026-10-05 23:04:09.032 | info | Metal | Finalize dfb: dfb_offset == base_offset: 0, dfb size: 0, return value: 0 (dataflow_buffer.cpp:1656) 2026-10-05 23:04:09.032 | info | Metal | Finalize dfb: dfb_offset == base_offset: 0, dfb size: 0, return value: 0 (dataflow_buffer.cpp:1656) (EngineCore_DP0 pid=150) 2026-10-05 23:04:12.142 | INFO | models.demos.gemma4.tt.serving_vllm:allocate_kv_cache_per_layer:212 - KV: full 4097 blocks x 64, sliding ring 18432 tokens; DRAM allocated 26.96 GB 2026-10-05 23:04:12.200 | info | Metal | Finalize dfb: dfb_offset == base_offset: 64, dfb size: 16, return value: 80 (dataflow_buffer.cpp:1656) 2026-10-05 23:04:12.200 | info | Metal | Finalize dfb: dfb_offset == base_offset: 0, dfb size: 0, return value: 0 (dataflow_buffer.cpp:1656) 2026-10-05 23:04:12.200 | info | Metal | Finalize dfb: dfb_offset == base_offset: 0, dfb size: 0, return value: 0 (dataflow_buffer.cpp:1656) 2026-10-05 23:04:12.200 | info | Metal | Finalize dfb: dfb_offset == base_offset: 0, dfb size: 0, return value: 0 (dataflow_buffer.cpp:1656) 2026-10-05 23:04:12.382 | info | Metal | Finalize dfb: dfb_offset == base_offset: 64, dfb size: 16, return value: 80 (dataflow_buffer.cpp:1656) 2026-10-05 23:04:12.382 | info | Metal | Finalize dfb: dfb_offset == base_offset: 0, dfb size: 0, return value: 0 (dataflow_buffer.cpp:1656) 2026-10-05 23:04:12.382 | info | Metal | Finalize dfb: dfb_offset == base_offset: 0, dfb size: 0, return value: 0 (dataflow_buffer.cpp:1656) 2026-10-05 23:04:12.382 | info | Metal | Finalize dfb: dfb_offset == base_offset: 0, dfb size: 0, return value: 0 (dataflow_buffer.cpp:1656) (EngineCore_DP0 pid=150) INFO 10-05 23:04:43 [prefix_resume.py:79] Prefix resume: chunk 16384, block 64, max snapshots 4 (EngineCore_DP0 pid=150) 2026-10-05 23:04:43.095 | INFO | models.demos.gemma4.tt.serving_vllm:warmup_model_prefill:241 - prefill warmup 31.0s (EngineCore_DP0 pid=150) 2026-10-05 23:04:43.095 | INFO | models.demos.gemma4.tt.single_user:_capture:254 - decode trace (re)capture at pos 16: ['LogitsDecoder', 'SpecGreedyDecoder'] (EngineCore_DP0 pid=150) 2026-10-05 23:04:43.279 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:43.279 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:43.314 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:43.315 | INFO | tracy:signpost:49 - gemma4_lm_head 2026-10-05 23:04:43.345 | warning | Metal | Allocating device buffers is unsafe due to the existence of an active trace. These buffers may be corrupted once a trace is executed. (allocator.cpp:123) 2026-10-05 23:04:43.506 | info | Metal | Finalize dfb: dfb_offset == base_offset: 64, dfb size: 16, return value: 80 (dataflow_buffer.cpp:1656) 2026-10-05 23:04:43.506 | info | Metal | Finalize dfb: dfb_offset == base_offset: 0, dfb size: 0, return value: 0 (dataflow_buffer.cpp:1656) 2026-10-05 23:04:43.506 | info | Metal | Finalize dfb: dfb_offset == base_offset: 0, dfb size: 0, return value: 0 (dataflow_buffer.cpp:1656) 2026-10-05 23:04:43.506 | info | Metal | Finalize dfb: dfb_offset == base_offset: 0, dfb size: 0, return value: 0 (dataflow_buffer.cpp:1656) (EngineCore_DP0 pid=150) 2026-10-05 23:04:43.506 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:43.507 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:43.576 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:43.576 | INFO | tracy:signpost:49 - gemma4_lm_head 2026-10-05 23:04:43.801 | info | Metal | Finalize dfb: dfb_offset == base_offset: 64, dfb size: 16, return value: 80 (dataflow_buffer.cpp:1656) 2026-10-05 23:04:43.801 | info | Metal | Finalize dfb: dfb_offset == base_offset: 0, dfb size: 0, return value: 0 (dataflow_buffer.cpp:1656) 2026-10-05 23:04:43.801 | info | Metal | Finalize dfb: dfb_offset == base_offset: 0, dfb size: 0, return value: 0 (dataflow_buffer.cpp:1656) 2026-10-05 23:04:43.801 | info | Metal | Finalize dfb: dfb_offset == base_offset: 0, dfb size: 0, return value: 0 (dataflow_buffer.cpp:1656) (EngineCore_DP0 pid=150) 2026-10-05 23:04:43.801 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:43.802 | INFO | tracy:signpost:49 - gemma4_lm_head 2026-10-05 23:04:43.901 | info | Metal | Finalize dfb: dfb_offset == base_offset: 64, dfb size: 16, return value: 80 (dataflow_buffer.cpp:1656) 2026-10-05 23:04:43.901 | info | Metal | Finalize dfb: dfb_offset == base_offset: 0, dfb size: 0, return value: 0 (dataflow_buffer.cpp:1656) 2026-10-05 23:04:43.901 | info | Metal | Finalize dfb: dfb_offset == base_offset: 0, dfb size: 0, return value: 0 (dataflow_buffer.cpp:1656) 2026-10-05 23:04:43.901 | info | Metal | Finalize dfb: dfb_offset == base_offset: 0, dfb size: 0, return value: 0 (dataflow_buffer.cpp:1656) (EngineCore_DP0 pid=150) 2026-10-05 23:04:43.901 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:43.902 | INFO | tracy:signpost:49 - gemma4_lm_head 2026-10-05 23:04:43.996 | info | Metal | Finalize dfb: dfb_offset == base_offset: 64, dfb size: 16, return value: 80 (dataflow_buffer.cpp:1656) 2026-10-05 23:04:43.996 | info | Metal | Finalize dfb: dfb_offset == base_offset: 0, dfb size: 0, return value: 0 (dataflow_buffer.cpp:1656) 2026-10-05 23:04:43.996 | info | Metal | Finalize dfb: dfb_offset == base_offset: 0, dfb size: 0, return value: 0 (dataflow_buffer.cpp:1656) 2026-10-05 23:04:43.996 | info | Metal | Finalize dfb: dfb_offset == base_offset: 0, dfb size: 0, return value: 0 (dataflow_buffer.cpp:1656) (EngineCore_DP0 pid=150) 2026-10-05 23:04:43.996 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:43.997 | INFO | tracy:signpost:49 - gemma4_lm_head 2026-10-05 23:04:44.099 | info | Metal | Finalize dfb: dfb_offset == base_offset: 64, dfb size: 16, return value: 80 (dataflow_buffer.cpp:1656) 2026-10-05 23:04:44.099 | info | Metal | Finalize dfb: dfb_offset == base_offset: 0, dfb size: 0, return value: 0 (dataflow_buffer.cpp:1656) 2026-10-05 23:04:44.099 | info | Metal | Finalize dfb: dfb_offset == base_offset: 0, dfb size: 0, return value: 0 (dataflow_buffer.cpp:1656) 2026-10-05 23:04:44.099 | info | Metal | Finalize dfb: dfb_offset == base_offset: 0, dfb size: 0, return value: 0 (dataflow_buffer.cpp:1656) (EngineCore_DP0 pid=150) 2026-10-05 23:04:44.099 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:44.100 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:44.161 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:44.161 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:44.235 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:44.236 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:44.306 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:44.306 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:44.382 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:44.383 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:44.460 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:44.461 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:44.534 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:44.535 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:44.622 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:44.623 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:44.661 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:44.661 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:44.746 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:44.747 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:44.803 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:44.803 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:44.914 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:44.914 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:44.957 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:44.957 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:45.001 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:45.001 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:45.045 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:45.045 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:45.089 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:45.089 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:45.158 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:45.158 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:45.233 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:45.234 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:45.312 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:45.312 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:45.391 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:45.391 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:45.471 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:45.472 | INFO | tracy:signpost:49 - gemma4_lm_head (EngineCore_DP0 pid=150) 2026-10-05 23:04:45.507 | INFO | models.demos.gemma4.tt.serving_vllm:warmup_model_decode:250 - decode trace capture 2.4s (EngineCore_DP0 pid=150) INFO 10-05 23:04:45 [core.py:286] init engine (profile, create kv cache, warmup model) took 36.80 seconds (EngineCore_DP0 pid=150) WARNING 10-05 23:04:47 [scheduler.py:166] Using custom scheduler class vllm_tt_plugin.scheduler.TTScheduler. This scheduler interface is not public and compatibility may not be maintained. (EngineCore_DP0 pid=150) INFO 10-05 23:04:47 [core.py:126] Scheduler class: TTScheduler, async_scheduling=False (EngineCore_DP0 pid=150) INFO 10-05 23:04:47 [prefix_resume.py:253] Prefix resume attached to the KV cache manager (EngineCore_DP0 pid=150) INFO 10-05 23:04:48 [vllm.py:698] Asynchronous scheduling is disabled. (EngineCore_DP0 pid=150) WARNING 10-05 23:04:48 [vllm.py:743] Inductor compilation was disabled by user settings, optimizations settings that are only active during inductor compilation will be ignored. (EngineCore_DP0 pid=150) INFO 10-05 23:04:48 [platform.py:796] Automatic prefix caching is enabled (APIServer pid=1) INFO 10-05 23:04:48 [api_server.py:495] Supported tasks: ['generate'] (APIServer pid=1) INFO 10-05 23:04:48 [parser_manager.py:202] "auto" tool choice has been enabled. (APIServer pid=1) WARNING 10-05 23:04:48 [model.py:1351] Default vLLM sampling parameters have been overridden by the model's `generation_config.json`: `{'temperature': 1.0, 'top_k': 64, 'top_p': 0.95}`. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`. (APIServer pid=1) INFO 10-05 23:04:48 [parser_manager.py:202] "auto" tool choice has been enabled. (APIServer pid=1) INFO 10-05 23:04:48 [serving.py:185] Warming up chat template processing... (APIServer pid=1) INFO 10-05 23:04:48 [hf.py:318] Detected the chat template content format to be 'openai'. You can set `--chat-template-content-format` to override this. (APIServer pid=1) INFO 10-05 23:04:48 [serving.py:210] Chat template warmup completed in 71.5ms (APIServer pid=1) INFO 10-05 23:04:48 [parser_manager.py:202] "auto" tool choice has been enabled. (APIServer pid=1) INFO 10-05 23:04:48 [api_server.py:500] Starting vLLM API server 0 on http://0.0.0.0:8000 (APIServer pid=1) INFO 10-05 23:04:48 [launcher.py:38] Available routes are: (APIServer pid=1) INFO 10-05 23:04:48 [launcher.py:47] Route: /openapi.json, Methods: HEAD, GET (APIServer pid=1) INFO 10-05 23:04:48 [launcher.py:47] Route: /docs, Methods: HEAD, GET (APIServer pid=1) INFO 10-05 23:04:48 [launcher.py:47] Route: /docs/oauth2-redirect, Methods: HEAD, GET (APIServer pid=1) INFO 10-05 23:04:48 [launcher.py:47] Route: /redoc, Methods: HEAD, GET (APIServer pid=1) INFO 10-05 23:04:48 [launcher.py:47] Route: /metrics, Methods: GET (APIServer pid=1) INFO: Started server process [1] (APIServer pid=1) INFO: Waiting for application startup. (APIServer pid=1) INFO: Application startup complete.