@hackernews@die-partei.social

High-performance inference of OpenAI's Whisper automatic speech recognition (ASR) model:

Plain C/C++ implementation without dependencies
Apple silicon first-class citizen - optimized via ARM NEON, Accelerate framework and Core ML
AVX intrinsics support for x86 architectures
VSX intrinsics support for POWER architectures
Mixed F16 / F32 precision
4-bit and 5-bit integer quantization support
Low memory usage (Flash Attention)
Zero memory allocations at runtime
Runs on the CPU
Partial GPU support for NVIDIA via cuBLAS
C-style API
+++++++++++++++++++++++++++++++++++++++++
What's Changed
Updated escape_double_quotes() Function by @tauseefmohammed2 in #776
examples : add missing #include by @pH5 in #798
Flush upon finishing inference by @tarasglek in #811
Escape quotes in csv output by @laytan in #815
C++11style by @wuyudi in #768
Optionally allow a Core ML build of Whisper to work with or without Core ML models by @Canis-UK in #812
add some tips about in the readme of the android project folder by @Zolliner in #816
whisper: Use correct seek_end when offset is used by @ThijsRay in #833
ggml : fix 32-bit ARM NEON by @ggerganov in #836
Add CUDA support via cuBLAS by @ggerganov in #834
Integer quantisation support by @ggerganov in #540