AI-based Bee Detection and Tracking

 

Usage of Mixtile Edge 2 Kit in agriculture: Bee detection, which can be essential for the health and survival of bees

 

Things used in this project

Hardware components      —      Mixtile Edge 2 Kit ×1

Software apps and online services     —    Snappy Ubuntu Core

 

Story

Mixtile is professional IoT hardware solution provider specialized in Linux and Android-based embedded systems.

Mixtile Edge 2 Kit is high-performance ARM single board computer. It comes in variants of 2GB of LPDDR4 DRAM and 16GB eMMC Flash storage, or 4GB of LPDDR4 DRAM and 32GB eMMC Flash storage. This single board computer comes with preinstalled Android 11, and it runs Ubuntu Linux operating system in Android container. It comes with large connectivity options (Bluetooth, 4G/5G Cellular, GPS, and Lora, Zigbee and Z-Wave). For those, you will need module, but it comes with default onboard Wi-Fi connectivity, Gigabit Ethernet Port (RJ45) and Serial Port (RS-485). Because it comes with RS-485 port, which is industrial standard, and it comes within a strong metal case, it seems to me that it can be really used in industrial projects. I used official Raspberry Pi 5 power supply in order to power up my Mixtile Edge 2 Kit.

So, an idea came to me why not to use it in agriculture, bee detection, which can be essential for health and survival of bees.

This project will cover setting up Mixtile Edge 2 Kit, and custom photo dataset form video in order to train custom YOLOv5 bee detection model.

YOLOv5 models must be trained on labelled data in order to learn classes of objects in that data.

I gathered data from video and trained model on my PC.

To train a model, I used python and typed in command line:

python train.py –img 640 –batch 16 –epochs 3 –data coco128.yaml –weights best.pt

My training results are summarized in the following table:

From this table you can see that images are divided into 4 detection classes:

  • Bee
  • Drone
  • Pollenbee
  • Queen

I used 100 images for each class (400 in total) and 1000 labels total.

Example for each class is summarized in a table below:

Bee Classes

Getting started

First, I will write about software part of the project, and later on steps of starting the recognition.

1. What is YOLOv5?

If you have been in the field of machine learning and deep learning for some time now, there is a high chance that you have already heard about YOLO. YOLO is short for You Only Look Once. It is a family of single-stage deep learning-based object detectors. It was written using Python language, and the framework used is PyTorch.

To ease control, I connected usb mouse to the one of three Mixtile Edge 2 Kit USB3 port. I used Ubuntu Linux for this project. Ubuntu on container is installed in Android system of Mixtile Edge 2 Kit by default. When you boot Mixtile Edge 2 Kit, you get Android OS. Since I wanted to access Edge 2 Kit remotely, and get easier control, I installed droidVNC server from this link:

https://f-droid.org/en/packages/net.christianbeier.droidvnc_ng/.

It is an Android VNC server using Android 5+ APIs. It does not require root access.

I started the VNC server, connected with VNC Viewer and I got the following Android 11 screen:

Android 11

After that, I installed SimpleSSHD from this link:

https://f-droid.org/en/packages/org.galexander.sshd/

SimpleSSHD is a SSH server Android app, based on Dropbear.It allows user access (user ssh) or full root access (by setting the login shell to /system/xbin/su) (if root is allowed).

After I installed SSH server, I connected to it via putty SSH terminal. Username and Password are root/root.

Com.hubware.ubuntu is ubuntu on a container and we are connected to it immidiately.

Now we are going to install required software.

First, you will need to upgrade Ubuntu by typing in the command: apt-get upgrade.

Second, I installed python by typing: apt-get install python.

You will also need pip, the package installer for Python.

2. Installing the YOLOv5 Environment

To start off we first clone the YOLOv5 repository and install dependencies. This will set up our programming environment to be ready to running object detection training and inference commands.

Install git: apt-get install git

Clone YOLOv5 repository:

git clone https://github.com/ultralytics/yolov5

Move to YOLOv5 folder:

cd yolov5

Install dependencies:

pip install -r requirements.txt

Wait some time to download and install all requirement packages, I waited 25 minutes, because there are a lot of python packages to install besides YOLOv5. YOLOv5 needs numpy package, scipy, OpenCV, etc.

The putty connection and installation process looks like below:

I transferred my model best.pt to the yolov5 installation folder via SCP, with MobaXterm.

You can simply download my model immidiate by typing:

wget https://github.com/sdizdarevic/beedetectionyolov5/raw/main/best.pt

Also, download original video by typing:

wget https://sdizdarevic.typepad.com/cr/bees-orig.mp4

Now, the final step is detection, and we are interested in the “result” content video.

python3 detect.py –weights best.pt –source

What this line does is simplified in below steps:

1. loads the model

2. loads the video

3. extracts frame from video one by one

4. applies prediction and show result

The process of detection looks like below:

In the last lines from last picture we can see the detected number of bees at any point in time.

If after starting the detection you get an error like below:

SyntaxError: Non-UTF-8 code starting with ‘xa3’ in file detect.py on line 1, but no encoding declared;

you need to delete the first line. You can do it with vi text editor:

vi detect.py.

The summarized short steps to follow are below:

Demonstrated videos are on urls with detection finished completely on Mixtile Edge 2 Kit. Output video is in folder runs/detect/exp2.

Original video:

Result video:

Last, but not less important: If you want to safely turn off your Mixtile Edge 2 Kit, I recommend you to install Shutdown (no Root) application: https://play.google.com/store/apps/details?id=com.samiadom.Shutdown&hl=en.

3.Conclusion:

After testing I found out that the Mixtile Edge 2 Kit is design with wide range of applications, from industrial applications, IOT devices, smart home automation, to more than capable AI and edge detection. It is low powered device, with a lot of built-in connectivity options.

I would like to thank amazing Mixtile people for creating this amazing peace of hardware and especially for sending me the Mixtile Edge 2 Kit. Also, Mixtile nurtures the open source values and software, and I belive more people and companies will be involved in making projects with this board.

All in all, I recommend this board for implementing types of projects I described here.

Code

Code steps BatchFile

Download 
git clone https://github.com/ultralytics/yolov5
cd yolov5
pip install -r requirements.txt
wget https://github.com/sdizdarevic/beedetectionyolov5/raw/main/best.pt 
wget https://sdizdarevic.typepad.com/cr/bees-orig.mp4 
python3 detect.py --weights best.pt --source bees-orig.mp4

Credits

Saudin Dizdarevic

I am an electrical engineer, and I am working as Project engineer in Oil and Gas industry. I like to work in my spare time on projects related to AI, Computer vision, Embedded systems and single board computers.