Image-Processing

By analysing the image with we receive from the camera, we can detect boats on the sea. To achieve this, different algorithms and approaches from the image processing and deep learning domain can be used.

S. Keller, K. Dierig, L. Range, "SearchWing - Messung des Energieverbrauchs eines Raspberry Pi und Charakterisierung des Kamersystems", Projektarbeit, WS 2020/21

Does JPEG image compression affect detection? See

Farhad Ghazvinian Zanjani, Svitlana Zinger, Bastian Piepers, Saeed Mahmoudpour, Peter Schelkens, and Peter H. N. de With "Impact of JPEG 2000 compression on deep convolutional neural networks for metastatic cancer detection in histopathological images," Journal of Medical Imaging 6(2), 027501 (24 April 2019). https://doi.org/10.1117/1.JMI.6.2.027501

https://gitlab.com/searchwing/development/boatdetectorcpp

Assumptions for the algorithms

By flying over the sea with 50-80km/h boats can be assumed to stand almost still in relation to the drone
Waves appear and dissapear over time
Approach to detect boats
- Detect parts in the image which dont change over time
- Redect parts by checking the same position in consecutive frames
- If parts look the same, save them
- If something is redetected over 3 frames, we can assume this could be a boat

Proposal / boat detector

Detection of randomly looking objects in the water
- Classical image processing ( no deep learning)
- Processing chain depicted in the image
There false positives after inital detection
Calculation of the 3D Position of the Objects
- In world coordinates
- By using a camera calibration
Code optimized for ARM Platforms
- By using ARM related flags for OpenCV
- Alternative: Compute Library for even faster processing
Runtime
- 0,8 sec @ 8 MegaPixel(3240×2480 Pixel) @ Raspberry Pi 3

Tracking

Redetect boats in consecutive frames
- Get more information/images about the same boats
  - See image below
- Reduce false-positive rate
  - Valid detections only if we redetect the same boat
- Algorithms
  - Association Problem
    - Euclidean distance based cost matrix between each possible track and detection
    - Solve global neigherest neighbor assignement problem by using Hungarian Algorithm
  - Tracking
    - Different trackingmodels possible
    - Constant position
    - Kalman constant position
    - Kalman constant velocity

Output

Save boat images on harddisk
Metadata for each detection is saved in exif-datablock of each detected boat
- GPS Position
- GPS Time
Visualization of the detection in digikam

asd

ROS
- Module communication
- Coordinatesystem transformations
- Recording and playback of datasets
- Visualisation
MAVROS
- Communication with the drone via MAVLINK
OpenCV
- Imageprocessing
ARM Compute Library
- Imageprocessing