Raspberry PI 3 B+ as an Access point/bridge on a local wireless network

A Raspberry PI 3 B+ based setup is expected here. This is detailed in a previous blog.

O. Create a PI based Network Access point using static host (pi@navx.local)

Major reference that describes how to create a DHCP server based WiFi bridge is available here, http://ardupilot.org/dev/docs/making-a-mavlink-wifi-bridge-using-the-raspberry-pi.html

maxresdefault

Hostname & Setup: Two Raspberry Pi’s. One is the access point and other connects to it for communicating over Wireless LAN.

navx.local -> pi@192.168.42.1 
-> This Raspberry PI is the WiFi access-point/hotspot host

nava.local -> pi@192.168.42.13 
-> This Raspberry PI is configured to connect to the above access-point

A. Setup or configure WiFi SSID on Raspberry (pi@nava.local)

Link: https://www.raspberrypi.org/documentation/configuration/wireless/wireless-cli.md

sudo nano /etc/wpa_supplicant/wpa_supplicant.conf

Connect the raspberry Pi to the WiFi access point. Edit the wpa_supplicant file with the new SSID and Password.

Alternately you can use raspi-config

sudo nano /etc/wpa_supplicant/wpa_supplicant.conf
sudo ifdown wlan0
sudo ifup wlan0

B. Change the hostname using raspi-config or using /etc/hosts & /etc/hostname file  (pi@nava.local & pi@navx.local)

Install avahi with the following commands on all the Raspberry Pi’s: https://www.howtogeek.com/167190/how-and-why-to-assign-the-.local-domain-to-your-raspberry-pi/

sudo apt-get install avahi-daemon
# Update boot startup for avahi-daemon
sudo insserv avahi-daemon
sudo update-rc.d avahi-daemon defaults

Install Bonjour on windows for access, discovery and then configure IPv6 on raspberry PI’s
https://peterlaszlo.wordpress.com/2013/06/27/bonjour-avahi-rpi-windows/

# Enable IPv6 on RPi 
sudo modprobe ipv6

Add ip6 entry on a new line in /etc/modules file

# Apply the new configuration with:
sudo /etc/init.d/avahi-daemon restart

Raspberry PI’s should now be addressable from other machines as navx.local,

ssh pi@navx.local
ssh pi@nava.local

 

C. Stop the static IP and connect back to WiFi on wlan0 (pi@navx.local)

In case you are connected to another WiFi with internet access before moving to this configuration. A quick way to connect back to that WiFi is below.

Disable, stop DHCP server and reboot

sudo update-rc.d hostapd disable
sudo update-rc.d isc-dhcp-server disable
sudo service hostapd stop
sudo service isc-dhcp-server stop

Move the previous network config to the current setup

cp /etc/network/interfaces.backup /etc/network/interfaces

Revert the interfaces file on Rpi rasbian by copying the content of backup to the interfaces. The backed up content is shown below.

(cv3) pi@navx:~ $ cat interface.dyninternet.backup
# interfaces(5) file used by ifup(8) and ifdown(8)
# Please note that this file is written to be used with dhcpcd
# For static IP, consult /etc/dhcpcd.conf and 'man dhcpcd.conf'

# Include files from /etc/network/interfaces.d:
source-directory /etc/network/interfaces.d

auto lo
iface lo inet loopback

iface eth0 inet manual

allow-hotplug intwifi0
iface intwifi0 inet manual
    wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf

allow-hotplug wlan0
iface wlan0 inet manual
    wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf

allow-hotplug wlan1
iface wlan1 inet manual
    wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf

D. Start the static IP configured DHCP server for the access point (pi@navx.local)

Start the DHCP server and enable the service

sudo service hostapd start
sudo service isc-dhcp-server start

sudo update-rc.d hostapd enable
sudo update-rc.d isc-dhcp-server enable

sudo reboot

Scan the wlan0 for SSID “NavxStation

sudo iwlist wlan0 scanning essid NavxStation

Connect from the raspberry (pi@nava.local) as described in section A

 

E. Provide fixed IP address on the host dhcp server (pi@navx.local)

https://blog.monotok.org/setup-raspberry-pi-dhcp-server/

The “HWaddr” or “ether” value is the MAC address. In this example say “c7:35:ce:fd:8e:a1

ifconfig wlan0

Edit the /etc/dhcp/dhcpd.conf file and add the following towards the end for fixed assignment.

host machine1_nava {
  hardware ethernet XX:XX:XX:XX:XX:XX;
  fixed-address 192.168.42.13;
}

Check the currently leased connections

cat /var/lib/dhcp/dhcpd.leases

Also, you can verify connected devices using

sudo iw dev wlan0 station dump 
sudo arp

 

 

 

Advertisements

Brokerless ZeroMq to share live image/video data stream from Raspberry PI B+ (Uses OpenCV)

Makes sure you have the following :

  1. OpenCV 3.x compiled and installed on Rasbperry Pi
  2. Raspberry Pi B+ with Raspbian and a USB webcam attached

ZeroMQ

Read through ZeroMQ in 100 words for a brief description

zmq

Installationhttp://zeromq.org/bindings:python

 

Code

Now lets go through the simple code I wrote for publishing and subscribing to live webcam stream from a raspberry PI to a workstation.

 

Make sure you have the dependencies and imports as below

import os, sys, datetime
import json, base64

import cv2
import zmq
import numpy as np
import imutils
from imutils.video import FPS

 

“”” Publish “””

  • In this piece of code we are creating a Zero MQ context preparing to send data to ‘tcp://localhost:5555’
  • The opencv API for camera is used to capture a frame from the device in 640×480 size
  • FPS module is used from opencv to estimate the frame rate for this capture
  • The byte buffer read from the webcam is encoded and sent as a string over the Zero MQ TCP socket connection
  • Continue to send each buffer out on the TCP socket
def pubVideo(config):
    context = zmq.Context()
    footage_socket = context.socket(zmq.PUB)
    # 'tcp://localhost:5555'
    ip = "127.0.0.1"
    port = 5555
    target_address = "tcp://{}:{}".format(ip, port) 
    print("Publish Video to ", target_address)
    footage_socket.connect(target_address)
    impath = 0 # For the first USB camera attached
    camera = cv2.VideoCapture(impath)  # init the camera
    camera.set(cv2.CAP_PROP_FRAME_WIDTH, 640)
    camera.set(cv2.CAP_PROP_FRAME_HEIGHT, 480)
    print("Start Time: ", datetime.datetime.now())
    fps = FPS().start()
    while True:
        try:
            buffer = capture(config, camera)
            if not isinstance(buffer, (list, tuple, np.ndarray)):
                break
            buffer_encoded = base64.b64encode(buffer)
            footage_socket.send_string(buffer_encoded.decode('ascii'))
            # Update the FPS counter
            fps.update()
            cv2.waitKey(1)
        except KeyboardInterrupt:
            # stop the timer and display FPS information
            fps.stop()
            print("[INFO] elasped time: {:.2f}".format(fps.elapsed()))
            print("[INFO] approx. FPS: {:.2f}".format(fps.fps()))
            camera.release()
            cv2.destroyAllWindows()
            print("\n\nBye bye\n")
            break
    print("End Time: ", datetime.datetime.now())

 

“”” Subscribe “””

  • The ZeroMQ subscriber is listening on ‘tcp://*:5555’ 
  • As the string is received its decoded and converted to image using OpenCv
  • We use OpenCV to visualize this frame in a window
  • Every frame sent over the ZeroMQ TCP socket is visualized and appears as a live video stream
def subVideo(config):
    context = zmq.Context()
    footage_socket = context.socket(zmq.SUB)
    port = 5555
    bind_address = "tcp://*:{}".format(port) # 'tcp://*:5555'
    print("Subscribe Video at ", bind_address)
    footage_socket.bind(bind_address)
    footage_socket.setsockopt_string(zmq.SUBSCRIBE, str(''))
    while True:
        try:
            frame = footage_socket.recv_string()
            img = base64.b64decode(frame)
            npimg = np.fromstring(img, dtype=np.uint8)
            source = cv2.imdecode(npimg, 1)
            cv2.imshow("image", source)
            cv2.waitKey(1)
        except KeyboardInterrupt:
            cv2.destroyAllWindows()
            print("\n\nBye bye\n")
            break

Github: https://github.com/vishwakarmarhl/cnaviz/blob/master/imcol/CvZmq.py

 

Run

In the github code above, you can run the test as follows,

PUB: Webcam source machine

python CvZmq.py pub --pubip=127.0.0.1 --pubport=5555

SUB: Target visualization machine

python CvZmq.py sub --subport=5555

 

Navio2 with Raspberry Pi 3 B+ for the Ardupilot flight controller setup

Load the Raspberry Pi Image provided by Emlid which has ROS and ardupilot pre-installed.

Controller Setup

Component/Part Name Documentation/Link Description
NAVIO2 Kit Ardupilot Navio2 Overview Sensor HAT for Pi
CanaKit Raspberry Pi 3 B+ Pi & Navio2 Setup Compute for flight
DJI F330 Flamewheel (or similar ARF Kit) Copter Assembly guide Frames, Motors, ESCs, Propellers
Radio Controller (Transmitter) Review of the RC products RC Transmitter
ELP USB FHD01M-L36 Camera ELP USB Webcam 2MP

IMG_20180821_151617446

Ardupilot

Verify 
(cv2) pi@nava:~/workspace/cnaviz/imcol $ ps -eaf | grep ardu
root 1909 1 0 16:36 ? 00:00:00 /bin/sh -c /usr/bin/arducopter $TELEM1 $TELEM2
root 1910 1909 15 16:36 ? 00:15:48 /usr/bin/arducopter -A udp:172.31.254.175:14550

Examples

Setup a Python 2 environment and clone Navio 2 repository

sudo apt-get install build-essential libi2c-dev i2c-tools python-dev libffi-dev
mkvirtualenv cv2 -p python2
pip install smbus-cffi
git clone https://github.com/emlid/Navio2.git
cd Navio2
Run tests
(cv2) pi@nava:~/Navio2/Python $ emlidtool test
2018-08-20 19:03:23 nava root[2337] INFO mpu9250: Passed
2018-08-20 19:03:23 nava root[2337] INFO adc: Passed
2018-08-20 19:03:23 nava root[2337] INFO rcio_status_alive: Passed
2018-08-20 19:03:23 nava root[2337] INFO lsm9ds1: Passed
2018-08-20 19:03:23 nava root[2337] INFO gps: Passed
2018-08-20 19:03:23 nava root[2337] INFO ms5611: Passed
2018-08-20 19:03:23 nava root[2337] INFO pwm: Passed
2018-08-20 19:03:23 nava root[2337] INFO rcio_firmware: Passed
Ardupilot should be stopped while running the Navio2 tests
sudo systemctl stop arducopter
Barometer
(cv2) pi@nava:~/Navio2/Python $ python Barometer.py
Temperature(C): 39.384754 Pressure(millibar): 1010.329778
Temperature(C): 39.333014 Pressure(millibar): 1010.368464
Accelerometer
(cv2) pi@nava:~/Navio2/Python $ python AccelGyroMag.py -i mpu
Selected: MPU9250
Connection established: True
Acc: -2.442 +9.428 +0.958 Gyr: -0.030 +0.011 -0.010 Mag: -3489.829 +30.680 +0.000
Acc: -2.504 +9.596 +1.063 Gyr: -0.023 +0.004 -0.012 Mag: -55.946 +6.677 +31.255
Acc: -2.346 +9.495 +0.924 Gyr: -0.023 +0.007 -0.007 Mag: -57.394 +5.955 +31.255
Acc: -2.370 +9.567 +1.020 Gyr: -0.030 +0.006 -0.014 Mag: -55.765 +6.497 +30.731
GPS
(cv2) pi@nava:~/Navio2/Python $ python GPS.py
gpsFix=0
Longitude=0 Latitude=0 height=0 hMSL=-17000 hAcc=4294967295 vAcc=4082849024
gpsFix=0
Longitude=0 Latitude=0 height=0 hMSL=-17000 hAcc=4294967295 vAcc=4083043328
ADC
(cv2) pi@nava:~/Navio2/Python $ python ADC.py
A0: 5.0100V A1: 0.0440V A2: 0.0160V A3: 0.0160V A4: 0.0180V A5: 0.0220V
A0: 5.0370V A1: 0.0440V A2: 0.0180V A3: 0.0140V A4: 0.0160V A5: 0.0240V
A0: 5.0370V A1: 0.0440V A2: 0.0160V A3: 0.0140V A4: 0.0160V A5: 0.0240V
LED
(cv2) pi@nava:~/Navio2/Python $ sudo python LED.py
LED is yellow
LED is green
LED is cyan
LED is blue
LED is magenta
LED is red
LED is yellow
LED is green
LED is cyan

 

Quick Apt Repository way – NVIDIA CUDA 9.x on Ubuntu 18.04 LST installation

The same NVIDIA CUDA 9.1 setup on Ubuntu 18.04 LST using the aptitude repository. However this appears to work and is simple to work with. Reference is taken from this askubuntu discussion.

Lookup the solution to the Nouveau issue from this blogpost

sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt update
sudo ubuntu-drivers autoinstall
sudo reboot

Now install the CUDA toolkit

sudo apt install g++-6
sudo apt install gcc-6
sudo apt install nvidia-cuda-toolkit gcc-6

Screenshot from 2018-07-13 14-18-16

Screenshot from 2018-07-13 14-16-00

Run the installer

root@wind:~/Downloads# ./cuda_9.1.85_387.26_linux --override

Screenshot from 2018-07-13 14-27-36.png

Screenshot from 2018-07-13 14-28-43

Setup the environment variables

# Environment variables
export PATH=/usr/local/cuda-9.1/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/cuda-9.1/lib64 
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/cuda-9.1/lib

Provide the soft link for the gcc-6 compiler

sudo ln -s /usr/bin/gcc-6 /usr/local/cuda/bin/gcc
sudo ln -s /usr/bin/g++-6 /usr/local/cuda/bin/g++
sudo reboot

Test

cd ~/NVIDIA_CUDA-9.1_Samples/
make -j4

Upon completion of the compilation test using device query binary

$ cd ~/NVIDIA_CUDA-9.1_Samples/bin/x86_64/linux/release
$ ./deviceQuery

Screenshot from 2018-07-13 14-41-49.png

$ sudo bash -c "echo /usr/local/cuda/lib64/ > /etc/ld.so.conf.d/cuda.conf"
$ sudo ldconfig

DONE

NVIDIA CUDA 9.x on Ubuntu 18.04 LST installation

Guide

An installation guide to take you through the NVIDIA graphics driver as well as CUDA toolkit setup on an Ubuntu 18.04 LTS.

A. Know your cards

Verify what graphics card you have on your machine

rahul@karma:~$ lspci | grep VGA
04:00.0 VGA compatible controller: 
NVIDIA Corporation GM204 [GeForce GTX 970] (rev a1)
rahul@karma:~$ sudo lshw -C video
 *-display 
 description: VGA compatible controller
 product: GM204 [GeForce GTX 970]
 vendor: NVIDIA Corporation
 physical id: 0
 bus info: pci@0000:04:00.0
 version: a1
 width: 64 bits
 clock: 33MHz
 capabilities: pm msi pciexpress vga_controller bus_master cap_list rom
 configuration: driver=nouveau latency=0
 resources: irq:30 memory:f2000000-f2ffffff memory:e0000000-efffffff memory:f0000000-f1ffffff ioport:2000(size=128) memory:f3080000-f30fffff

Download the right driver

downloaded the Version 390.67 for GeForce GTX 970

Screenshot from 2018-07-12 17-15-34.png

B. Nouveau problem kills your GPU rush

Hoever there are solutions available

Here is what worked for me

  1. remove all nvidia packages ,skip this if your system is fresh installed
    sudo apt-get remove nvidia* && sudo apt autoremove
    
  2. install some packages for build kernel:
    sudo apt-get install dkms build-essential linux-headers-generic
    
  3. now block and disable nouveau kernel driver:
    sudo vim /etc/modprobe.d/nvidia-installer-disable-nouveau.conf
    

Insert follow lines to the nvidia-installer-disable-nouveau.conf:

blacklist nouveau
blacklist lbm-nouveau
options nouveau modeset=0
alias nouveau off
alias lbm-nouveau off

save and exit.

  1. Disable the Kernel nouveau by typing the following commands(nouveau-kms.conf may not exist,it is ok):
    rahul@wind:~$ echo options nouveau modeset=0 | sudo tee -a /etc/modprobe.d/nouveau-kms.conf
    options nouveau modeset=0
    
  2. build the new kernel by:
    rahul@wind:~$ sudo update-initramfs -u
    update-initramfs: Generating /boot/initrd.img-4.15.0-23-generic
    
  3. reboot
Run the Installer in run-level 3
$ sudo init 3 
$ sudo bash
$ ./NVIDIA-Linux-x86_64-390.67.run

Uninstall

More instruction on how to stop using the driver before uninstallation
sudo nvidia-installer –uninstall

C. NVIDIA X Server Settings

Install this from the ubuntu software center.
Screenshot from 2018-07-12 17-23-43.png

D. Start the CUDA related setup

We will need the CUDA toolkit 9.1 which is supported for the GTX 970 version with compute 3.0 capability. So download the local installer for Ubuntu.

Screenshot from 2018-07-13 13-55-24.png

Downloaded the “cuda_9.1.85_387.26_linux.run*” local installation file.

$ sudo add-apt-repository ppa:graphics-drivers/ppa
$ sudo apt install nvidia-cuda-toolkit gcc-6

Steps are taken from the CUDA 9.1 official documentation

  1. Perform the pre-installation actions.
  2.  Disable the Nouveau drivers. We did this in the above driver installation
  3. Reboot into text mode (runlevel 3). This can usually be accomplished by adding the number “3” to the end of the system’s kernel boot parameters. Change the runlevel ‘sudo init 3’, refer
  4. Verify that the Nouveau drivers are not loaded. If the Nouveau drivers are still loaded, consult your distribution’s documentation to see if further steps are needed to disable Nouveau.
  5. Run the installer and follow the on-screen prompts:
$ chmod +x cuda_9.1.85_387.26_linux
$ rahul@wind:~/Downloads$ ./cuda_9.1.85_387.26_linux --override

Screenshot from 2018-07-13 13-52-19.png

Since we already installed the Driver above we say NO in the NVIDIA accelerated graphic driver installation question.

Screenshot from 2018-07-13 13-54-20.png

This will install the CUDA stuff in the following locations

  • CUDA Toolkit /usr/local/cuda-9.1
  • CUDA Samples $(HOME)/NVIDIA_CUDA-9.1_Samples

We can verify the graphic card using the NVIDIA-SMI command.

Screenshot from 2018-07-12 20-02-08

Uninstallation

cd /usr/local/cuda-9.1/bin
sudo ./uninstall_cuda_9.1.pl

 

E. Environment Variables

rahul@wind:~$ vim ~/.bashrc

# Add the following to the environment variables
export PATH=/usr/local/cuda-9.1/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/cuda-9.1/lib64 
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/cuda-9.1/lib

rahul@wind:~$ source ~/.bashrc
rahul@wind:~$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2018 NVIDIA Corporation
Built on Tue_Jun_12_23:07:04_CDT_2018
Cuda compilation tools, release 9.1, 

 

F. Test

Ensure you have the right driver versions

rahul@wind:$ cat /proc/driver/nvidia/version
NVRM version: NVIDIA UNIX x86_64 Kernel Module 390.67 Fri Jun 1 04:04:27 PDT 2018
GCC version: gcc version 7.3.0 (Ubuntu 7.3.0-16ubuntu3)

Change directory to the NVIDIA CUDA Samples and compile them

rahul@wind:~/NVIDIA_CUDA-9.1_Samples$ make

Now run the device query test

rahul@wind:~/NVIDIA_CUDA-9.1_Samples/bin/x86_64/linux/release$ ./deviceQuery
./deviceQuery Starting...

CUDA Device Query (Runtime API) version (CUDART static linking)

 

END

 

Compile and Setup OpenCV 3.4.x on Ubuntu 18.04 LTS with Python Virtualenv for Image processing with Ceres, VTK, PCL

OpenCV: Open Source Computer Vision Library

Links

Documentation: https://docs.opencv.org/3.4.2/

OpenCV Source: https://github.com/opencv/opencv

OpenCV_Logo

A. Setup an external HDD/SSD for this setup

filesystem-partition-ubuntu-external-ssd

B. Environment (Ubuntu 18.04 LTS)

 

Python3 setup

Install the needed packages in a python virtualenv. Refer similar windows Anaconda setup or look at the ubuntu based info here

sudo apt-get install -y build-essential cmake unzip pkg-config 
sudo apt-get install -y ubuntu-restricted-extras
sudo apt-get install -y python3-dev python3-numpy
sudo apt-get install -y git python3-pip virtualenv
sudo pip3 install virtualenv
rahul@karma:~$ virtualenv -p /usr/bin/python3 cv3
Already using interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in /home/rahul/cv3/bin/python3
Also creating executable in /home/rahul/cv3/bin/python
Installing setuptools, pkg_resources, pip, wheel...
done.

Activate and Deactivate the python Environment

rahul@karma:~$ source ~/cv3/bin/activate
(cv3) rahul@karma:~$ python
Python 3.6.5 (default, Apr 1 2018, 05:46:30) 
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> print("Test",4*5)
Test 20
>>> exit()
(cv3) rahul@karma:~$ deactivate
rahul@karma:~$ 

Alternatively, a great way to use virtualenv is to use Virtualenvwrappers

sudo pip3 install virtualenv virtualenvwrapper

Add these to your ~/.bashrc file

# virtualenv and virtualenvwrapper
export WORKON_HOME=$HOME/.virtualenvs
export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3
source /usr/local/bin/virtualenvwrapper.sh

Now, run “source ~/.bashrc” to set the environment

Create a Virtual environment
rahul@karma:~$ mkvirtualenv cv3 -p python3
Already using interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in /home/rahul/.virtualenvs/cv3/bin/python3
Also creating executable in /home/rahul/.virtualenvs/cv3/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.
virtualenvwrapper.user_scripts creating /home/rahul/.virtualenvs/cv3/bin/predeactivate
virtualenvwrapper.user_scripts creating /home/rahul/.virtualenvs/cv3/bin/postdeactivate
virtualenvwrapper.user_scripts creating /home/rahul/.virtualenvs/cv3/bin/preactivate
virtualenvwrapper.user_scripts creating /home/rahul/.virtualenvs/cv3/bin/postactivate
virtualenvwrapper.user_scripts creating /home/rahul/.virtualenvs/cv3/bin/get_env_details
(cv3) rahul@karma:~$
Activate/Deactivate virtual env
rahul@karma:~$ workon cv3
(cv3) rahul@karma:~$ deactivate 
rahul@karma:~$

Install basic packages for the computer vision work.

(cv3) rahul@karma: pip install numpy scipy scikit-image scikit-learn  
pip install imutils pyzmq ipython matplotlib
pip install dronekit==2.9.1 future==0.15.2 monotonic==1.2 pymavlink==2.0.6

Java installation from this blog

sudo add-apt-repository ppa:linuxuprising/java
sudo apt update
sudo apt install oracle-java10-installer
sudo apt install oracle-java10-set-default sudo apt-get install ant
Packages needed for OpenCV and others
GTK support for GUI features, Camera support (libv4l), Media Support (ffmpeg, gstreamer) etc. Additional packages for image formats mostly downloaded form the ubuntu-restricted-extra repository

 

sudo apt-get install -y libjpeg-dev libpng-dev libtiff-dev ffmpeg
sudo apt-get install -y libjpeg8-dev libjasper-dev libpng12-dev libtiff5-dev
sudo apt-get install -y libavcodec-dev libavformat-dev libswscale-dev libv4l-dev
sudo apt-get install -y libxvidcore-dev libx264-dev libvorbis-dev
sudo apt-get install -y libgtk2.0-dev libgtk-3-dev ccache imagemagick
sudo apt-get install -y liblept5 leptonica-progs libleptonica-dev
sudo apt-get install -y qt5-default libgtk2.0-dev libtbb-dev
sudo apt-get install -y libatlas-base-dev gfortran libblas-dev liblapack-dev 
sudo apt-get install -y libdvd-pkg libgstreamer-plugins-base1.0-dev
sudo apt-get install -y libmp3lame-dev libtheora-dev
sudo apt-get install -y libxine2-dev libv4l-dev x264 v4l-utils
sudo apt-get install -y libopencore-amrnb-dev libopencore-amrwb-dev

# Optional dependencies
sudo apt-get install -y libprotobuf-dev protobuf-compiler
sudo apt-get install -y libgoogle-glog-dev libgflags-dev
sudo apt-get install -y libgphoto2-dev libeigen3-dev libhdf5-dev doxygen

 

VTK for SFM Modules

SFM setup: https://docs.opencv.org/3.4.2/db/db8/tutorial_sfm_installation.html

sudo apt-get install libxt-dev libglew-dev libsuitesparse-dev
sudo apt-get install tk8.5 tcl8.5 tcl8.5-dev tcl-dev

Ceres-Solver: http://ceres-solver.org/installation.html

# However, if you want to build Ceres as a *shared* library, 
# You must, add the following PPA:
sudo add-apt-repository ppa:bzindovic/suitesparse-bugfix-1319687
sudo apt-get update
sudo apt-get install libsuitesparse-dev
git clone https://ceres-solver.googlesource.com/ceres-solver
cd ceres-solver
mkdir build && cd build
export CXXFLAGS="-std=c++11" 
cmake ..
make -j4
make test
sudo make install

LAPACK

sudo apt-get install libblas-dev libblas-doc liblapacke-dev liblapack-doc

 

VTK Setup, https://gitlab.kitware.com/vtk/vtk.git

Configure and build with QT support

git clone git://vtk.org/VTK.git VTK
cd VTK
mkdir VTK-build
cd VTK-build
CXXFLAGS="-std=c++11" cmake ../ -DBUILD_SHARED_LIBS=ON -DBUILD_TESTING=ON \
-DCMAKE_BUILD_TYPE=Release \
-DQT_QMAKE_EXECUTABLE:PATH=/usr/bin/qmake \
-DVTK_Group_Qt:BOOL=ON \
-DBUILD_SHARED_LIBS:BOOL=ON \
-DVTK_WRAP_PYTHON=ON  \
-DPYTHON_EXECUTABLE=~/.virtualenvs/cv3/bin/python 
make -j4
sudo make install
$ cp -r ~/cv/VTK/VTK-build/lib/python3.6/site-packages/* ~/.virtualenvs/cv3/lib/python3.6/site-packages/
$ export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/lib:/usr/local/lib"
$ sudo ldconfig

FLANN

http://www.cs.ubc.ca/research/flann/#download
cd flann-1.8.4-src/ && mkdir build && cd build
cmake ..
make -j4 
sudo make install

PCL

Download: http://www.pointclouds.org/downloads/linux.html

sudo apt-get install -y libusb-1.0-0-dev libusb-dev libudev-dev
sudo apt-get install -y mpi-default-dev openmpi-bin openmpi-common 
sudo apt-get install -y libboost-all-dev libpcap-dev  sudo apt-get install -y libqhull* libgtest-dev sudo apt-get install -y freeglut3-dev pkg-config sudo apt-get install -y libxmu-dev libxi-dev sudo apt-get install -y mono-complete sudo apt-get install -y openjdk-8-jdk openjdk-8-jre
git clone https://github.com/PointCloudLibrary/pcl 
# https://github.com/PointCloudLibrary/pcl/archive/pcl-1.8.1.tar.gz 
cd pcl && mkdir build && cd build 
CXXFLAGS="-std=gnu++11" cmake -DBUILD_apps=ON \
 -DBUILD_apps_point_cloud_editor=ON \
 -DBUILD_apps_cloud_composer=ON \
 -DBUILD_apps_modeler=ON \
 -DBUILD_apps_3d_rec_framework=ON \
 -DBUILD_examples=ON ..
make -j8 
sudo make install

Official OpenCV installation

wget -O opencv.zip https://github.com/opencv/opencv/archive/3.4.2.zip
wget -O opencv_contrib.zip https://github.com/opencv/opencv_contrib/archive/3.4.2.zip
unzip opencv.zip
unzip opencv_contrib.zip

In case of Raspberry PI 3 B+, this blog worked for me.

Link: https://www.pyimagesearch.com/2016/04/18/install-guide-raspberry-pi-3-raspbian-jessie-opencv-3/

 

Configure OpenCV with CMake
$ cd ~/cv/opencv-3.4.2 && mkdir build && cd build
$ CXXFLAGS="-std=c++11" cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=/usr/local \
-D INSTALL_PYTHON_EXAMPLES=ON \
-D INSTALL_C_EXAMPLES=OFF \
-D BUILD_opencv_java=OFF \
-D OPENCV_EXTRA_MODULES_PATH=~/cv/opencv_contrib-3.4.2/modules \
-D WITH_TBB=ON \
-D WITH_V4L=ON \
-D WITH_QT=ON \
-D WITH_OPENGL=ON \
-D WITH_VTK=ON \
-D WITH_GTK3=ON \
-D PYTHON_EXECUTABLE=~/.virtualenvs/cv3/bin/python \
-D BUILD_EXAMPLES=ON ..
Screenshot from 2018-07-12 13-18-55

Make sure the Python 3 interpreter and other dependencies are configured correctly.

Compiling with CUDA (Setup instructions)

cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=/usr/local \
-D INSTALL_PYTHON_EXAMPLES=ON \
-D INSTALL_C_EXAMPLES=ON \
-D BUILD_NEW_PYTHON_SUPPORT=ON \
-D OPENCV_EXTRA_MODULES_PATH=~/cv/opencv_contrib-3.4.2/modules \
-D WITH_TBB=ON \
-D WITH_V4L=ON \
-D WITH_QT=ON \
-D WITH_OPENGL=ON \
-D WITH_VTK=ON \
-D WITH_GTK3=ON \
-D PYTHON_EXECUTABLE=~/.virtualenvs/cv3/bin/python \
-D WITH_CUDA=ON \
-D ENABLE_FAST_MATH=1 \
-D CUDA_FAST_MATH=1 \
-D WITH_CUBLAS=1 \
-D BUILD_EXAMPLES=ON ..

 

Compile, Install and Verify
(cv3) rahul@karma:~/cv/opencv-3.4.2/build$ make -j4
$ sudo make install
$ sudo sh -c 'echo "/usr/local/lib" >> /etc/ld.so.conf.d/opencv.conf'
$ sudo ldconfig
$ pkg-config --modversion opencv
Setup the cv shared libraries
(cv3) rahul@karma$ ls -l /usr/local/lib/python3.6/site-packages
total 5172
-rw-r--r-- 1 root staff 5292240 Jul 12 13:32 cv2.cpython-36m-x86_64-linux-gnu.so
# or use the find command 
$ find /usr/local/lib/ -type f -name "cv2*.so"
$ cd /usr/local/lib/python3.6/site-packages/
$ mv cv2.cpython-36m-x86_64-linux-gnu.so cv2.so
$ cd ~/.virtualenvs/cv3/lib/python3.6/site-packages/
$ ln -s /usr/local/lib/python3.6/site-packages/cv2.so cv2.so

C. Test

(cv3) rahul@karma:~$ python
Python 3.6.5 (default, Apr 1 2018, 05:46:30) 
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information
>>> import cv2
>>> cv2.__version__
'3.4.2'
>>> exit()
(cv3) rahul@karma:~$

Done.

Anaconda for your Image Processing, Machine Learning, Neural Networks, Computer Vision development environment using VS Code

Python is a great language and I will not go into explaining why it is so. Here is a brief setup for your development environment in case you are tinkering with computer vision problems and looking at learning neural network on your windows laptop.

Anaconda3 5.0

64 bit Download: https://www.anaconda.com/download

Install Anaconda with the default options.

  • Anaconda Navigator is a great place to look at your environment and activate them as per your need.
  • In case you want to have a Python 2x and 3x environment side by side, then you can create them in navigator. Here I have a base(root) setup with Python 3.6 and an additional Python 2.7 environment.
  • In order to use a particular environment you can click on that environment in the navigator or go to the Anaconda prompt and execute the following command
"(base)C:\Users\Karma>activate Py27"
  • To deactivate use
deactivate
  • To create a new environment use the following command:
(base)C:\Users\Karma>conda create -n Py27 python=2.7 anaconda

Anaconda-Navigator

Whenever you want to use a particular environment just go to the environments section and activate it. This will setup your python with the packages and version as configured in that environment.  In the screenshot above I have tensorflow in my base environment while its always better to have a separate environment for this.

In case you are using Cmder like me then go for this:

Considering where you have installed your Anaconda
> C:\Anaconda3\Scripts\activate.bat C:\Anaconda3
or
> C:\Users\Karma\Anaconda3\Scripts\activate.bat C:\Users\Karma\Anaconda3
> conda info --envs
> conda activate py27
> conda deactivate

Lets try to use package manager “conda” for the setup.

Run the following installation command on Anaconda Command Prompt which will open up showing prompt as (C:\Anaconda3) C:\Users\Karma>:

In order to find packages, you should look at the Anaconda repository ( https://anaconda.org/anaconda/repo )

# Adding the menpo channels and install opencv
conda install -c https://conda.binstar.org/menpo opencv
conda config --add channels menpo
conda install -c menpo opencv

# or directly use conda-forge
conda install -c conda-forge opencv

# Install packages
conda install numpy
conda install scipy
conda install matplotlib

# List packages
conda list

OpenCV

If the OpenCV installation did not go through then we can use the pre-built windows binaries maintained by,

Christoph Gohlke at https://www.lfd.uci.edu/~gohlke/pythonlibs/#opencv

Download File: You can remove these modules by using “pip uninstall <package>”

(base)λ pip install opencv_python-3.4.0-cp36-cp36m-win_amd64.whl
Processing c:\users\karma\downloads\opencv_python-3.4.0-cp36-cp36m-win_amd64.whl
Installing collected packages: opencv-python
Successfully installed opencv-python-3.4.0
(base)λ pip install opencv_python-3.4.0+contrib-cp36-cp36m-win_amd64.whl
Processing c:\users\karma\downloads\opencv_python-3.4.0+contrib-cp36-cp36m-win_amd64.whl
Installing collected packages: opencv-python
Successfully installed opencv-python-3.4.0+contrib

In my case I used SIFT and SURF implementations which were made available in the contrib packages.

Now, that we have packages set, lets test it out on the python interpreter interface,
Use the following commands on the python CLI.

import numpy as np
import cv2

TensorFlow

Instructions: https://www.tensorflow.org/install/install_windows

To install this package with conda run:
conda install -c conda-forge tensorflow

Version changes based on the repository you are trying to download from.

I typically use VS Code but if you like smooth scrolling go for Sublime.

In VS Code I use ms-python.python, tht13.python extensions to simplify my workspace.

VSCode-Python

Debugging is critical to work with any kind of code. So here is some configuration to get you started here.

  • Verify that the workspace settings.json file has the right python path
"python.pythonPath ": "C:\\Anaconda3\\python.exe"
  • Add a launch.json in your project .vscode folder with the following values
{
   "name": "Python",
   "type": "python",
   "pythonPath":"${config:python.pythonPath}", "request": "launch", "stopOnEntry": true, "console": "none", "program": "${file}", "cwd": "${workspaceFolder}", "debugOptions": [ "WaitOnAbnormalExit", "WaitOnNormalExit", "RedirectOutput" ] }
This will get you setup for debugging and here is how the debug interface would look like when you have put the breakpoints and stepped through the code.
VSCode-Python-Debug

Good Luck.