whisper.cpp

Running

whisper.cpp / ggml /src /ggml-cuda /template-instances /fattn-mma-f16-instance-ncols1_8-ncols2_4.cu

CUDA: optimize FA for GQA + large batches (llama/12014)

6662d54 10 months ago

330 Bytes

	// This file has been autogenerated by generate_cu_files.py, do not edit manually.

	#include "../fattn-mma-f16.cuh"

	DECL_FATTN_MMA_F16_CASE(64, 8, 4);
	DECL_FATTN_MMA_F16_CASE(80, 8, 4);
	DECL_FATTN_MMA_F16_CASE(96, 8, 4);
	DECL_FATTN_MMA_F16_CASE(112, 8, 4);
	DECL_FATTN_MMA_F16_CASE(128, 8, 4);
	DECL_FATTN_MMA_F16_CASE(256, 8, 4);