Accelerating TensorFlow Lite Micro on Embedded RISC-V Platforms via Multi-Hart Execution

This paper presents a systematic evaluation of performance gains from Hardware Multithreading in ML applications on an embedded RISC-V platform, using OpenMP C/C++ extensions applied to TensorFlow Lite Micro (TFLM). The target hardware includes a multi-threaded MIPS P8700 CPU core implemented on a ProFPGA VU19P platform. We focus on optimizing three computationally intensive TFLM kernels�Conv2D, DepthwiseConv2D, and Pooling�by applying thread-aware OpenMP scheduling policies. To evaluate the impact of thread-level parallelism we introduced tensor layout specific OpenMP strategies into out-of-box person detection example from TFLM repository, which runs reference kernels for MobileNet V1 model. These structured optimizations led to up to 40% reduction in inference latency on single-core dual-hart configuration, compared to unmodified baseline. Our findings underscore the benefits of hardware multithreading and provide reproducible strategies for real-time, low power embedded deep learning inference on resource constrained hardware.