i’m looking for a bird’s eye view guidance on what AI/LLM ASIC accelerator modules can be used in a BPI-R4 (or other BPI boards if the BPI-R4 is lacking something).
i’m new to all this, so pardon my ignorance!
my primary goal is to add object detection to Frigate with low power consumption, but if i can use the same module for running local LLM models with a reasonable performance, then it’d be even better.
i think these should/could work in a BPI-R4:
-
Coral (Google’s Edge TPU) Mini PCIe Accelerator | Coral
The USB version is compatible with the widest variety of hardware and does not require a driver on the host machine. However, it does lack the automatic throttling features of the other versions.
-
Hailo It seems to work in a BPI-M7 at least: Hailo AI module with Banana Pi | BananaPi Docs
-
any other?
i have some doubts/questions, though:
-
some of these modules can draw a lot of power temporarily (2A). would the BPI-R4 be able to supply it?
-
what about the software support? will these work with an OpenWrt 24.10.1?
-
what about local memory? i can’t find any info. or these TPUs have no local memory? do they use the system’s memory through the PCIe lanes? would the BPI-R4’s 4GB be enough for running the object detection (next to the frigate docker image, which is already running fine on my board)?
-
what about the USB based TPUs, e.g. the Coral? seems to have the same performance. or are they available at all?
-
would i need an M.2 or a mini PCIe version of the Coral for the BPI-R4? if both exist and work, then which one is better?