Anaconda for your Image Processing, Machine Learning, Neural Networks, Computer Vision development environment using VS Code

Python is a great language and I will not go into explaining why it is so. Here is a brief setup for your development environment in case you are tinkering with computer vision problems and looking at learning neural network on your windows laptop.

Anaconda3 5.0

64 bit Download: https://www.anaconda.com/download

Install Anaconda with the default options.

  • Anaconda Navigator is a great place to look at your environment and activate them as per your need.
  • In case you want to have a Python 2x and 3x environment side by side, then you can create them in navigator. Here I have a base(root) setup with Python 3.6 and an additional Python 2.7 environment.
  • In order to use a particular environment you can click on that environment in the navigator or go to the Anaconda prompt and execute the following command

“(base)C:\Users\Karma>activate Py27”

Anaconda-Navigator

Whenever you want to use a particular environment just go to the environments section and activate it. This will setup your python with the packages and version as configured in that environment.  In the screenshot above I have tensorflow in my base environment while its always better to have a separate environment for this.

In case you are using Cmder like me then go for this:

> C:\Anaconda3\Scripts\activate.bat C:\Anaconda3
> conda info --envs
> conda activate py27

Lets try to use package manager “conda” for the setup.

Run the following installation command on Anaconda Command Prompt which will open up showing prompt as (C:\Anaconda3) C:\Users\Karma>:

# Adding the menpo channels and install opencv
conda install -c https://conda.binstar.org/menpo opencv
conda config --add channels menpo
conda install -c menpo opencv

# or directly use conda-forge
conda install -c conda-forge opencv

# Install packages
conda install numpy
conda install scipy
conda install matplotlib

# List packages
conda list

OpenCV

If the OpenCV installation did not go through then we can use the pre-built windows binaries maintained by,

Christoph Gohlke at https://www.lfd.uci.edu/~gohlke/pythonlibs/#opencv

Download File: You can remove these modules by using “pip uninstall <package>”

(base)λ pip install opencv_python-3.4.0-cp36-cp36m-win_amd64.whl
Processing c:\users\karma\downloads\opencv_python-3.4.0-cp36-cp36m-win_amd64.whl
Installing collected packages: opencv-python
Successfully installed opencv-python-3.4.0
(base)λ pip install opencv_python-3.4.0+contrib-cp36-cp36m-win_amd64.whl
Processing c:\users\karma\downloads\opencv_python-3.4.0+contrib-cp36-cp36m-win_amd64.whl
Installing collected packages: opencv-python
Successfully installed opencv-python-3.4.0+contrib

In my case I used SIFT and SURF implementations which were made available in the contrib packages.

Now, that we have packages set, lets test it out on the python interpreter interface,
Use the following commands on the python CLI.

import numpy as np
import cv2

TensorFlow

Instructions: https://www.tensorflow.org/install/install_windows

To install this package with conda run:
conda install -c conda-forge tensorflow

I typically use VS Code but if you like smooth scrolling go for Sublime.

In VS Code I use ms-python.python, tht13.python extensions to simplify my workspace.

VSCode-Python

Debugging is critical to work with any kind of code. So here is some configuration to get you started here.

  • Verify that the workspace settings.json file has the right python path
"python.pythonPath ": "C:\\Anaconda3\\python.exe"
  • Add a launch.json in your project .vscode folder with the following values
{
   "name": "Python",
   "type": "python",
   "pythonPath":"${config:python.pythonPath}", "request": "launch", "stopOnEntry": true, "console": "none", "program": "${file}", "cwd": "${workspaceFolder}", "debugOptions": [ "WaitOnAbnormalExit", "WaitOnNormalExit", "RedirectOutput" ] }
This will get you setup for debugging and here is how the debug interface would look like when you have put the breakpoints and stepped through the code.
VSCode-Python-Debug

Good Luck.

Advertisements

Eclipse can get naggy and here are some fixes

 

1. Make changes to the plugins that gets loaded at the startup/shutdown

Preferences -> General -> Startup and Shutdown -> Deselect all plugins

2. Modify the eclipse.ini file to have a larger memory allocation

Use the following content in your eclipse.ini

 

-startup
plugins/org.eclipse.equinox.launcher_1.3.0.v20130327-1440.jar
--launcher.library
plugins/org.eclipse.equinox.launcher.win32.win32.x86_64_1.1.200.v20130807-1835
-product
org.eclipse.epp.package.jee.product
--launcher.defaultAction
openFile
--launcher.XXMaxPermSize
256M
-showsplash
org.eclipse.platform
--launcher.XXMaxPermSize
256m
--launcher.defaultAction
openFile
--launcher.appendVmargs
-vm
C:/Program Files/Java/jdk1.8.0_91/bin/javaw.exe
-vmargs
-Dosgi.requiredJavaVersion=1.6
-Xverify:none
-Xmn128m
-Xms1024m
-Xmx1024m
-Xss2m
-XX:PermSize=512m
-XX:MaxPermSize=512m
-XX:+UseParallelGC

 

 

 

Repository Management with Nexus 3 for your Mavenized project, including release and snapshot distribution

Like the Nexus documentation says;

Stop developing in the Dark Ages, read this book, and start using a repository manager. Trust us, once you start using a Nexus Repository Manager, you’ll wonder how you ever functioned without it.

Reference:

A. Download the archive from https://www.sonatype.com/download-oss-sonatype

B. Unzip it into a folder and run it as follows

cd ~\nexus\nexus-3.3.1-01\bin
nexus.exe /run 

If the log shows the following that means the server is up  
-------------------------------------------------
 Started Sonatype Nexus OSS 3.3.1-01
-------------------------------------------------

C. Server starts by default on http://localhost:8081

username: admin  
password: admin123 
Use the above credentials to login as the default administrator

D. Add the following configuration to the ~\USER_HOME\.m2\settings.xml

Make sure you remove the code tags before using this configuration, which is used here for wordpress content formatting only.

<settings>
	  <mirrors>
		<mirror>
		  <!--This sends everything else to /public -->
		  <id>nexus</id>
		  <mirrorOf>*</mirrorOf>
		  <url>http://localhost:8081/repository/maven-public/</url>
		</mirror>
	  </mirrors>
	  <profiles>
		<profile>
		  <id>nexus</id>
		  <!--Enable snapshots for the built in central repo to direct -->
		  <!--all requests to nexus via the mirror -->
		  <repositories>
			<repository>
			  <id>central</id>
			  <url>http://central</url>
			  <releases><enabled>true</enabled></releases>
			  <snapshots><enabled>true</enabled></snapshots>
			</repository>
		  </repositories>
		 <pluginRepositories>
			<pluginRepository>
			  <id>central</id>
			  <url>http://central</url>
			  <releases><enabled>true</enabled></releases>
			  <snapshots><enabled>true</enabled></snapshots>
			</pluginRepository>
		  </pluginRepositories>
		</profile>
	  </profiles>
	  <activeProfiles>
		<!--make the profile active all the time -->
		<activeProfile>nexus</activeProfile>
	  </activeProfiles>
	   <servers>
		<server>
		  <id>nexus</id>
		  <username>admin</username>
		  <password>admin123</password>
		</server>
	  </servers>
	</settings>

E. Release and snapshot artifacts should be configured in the projects pom as distributionManagement

  <distributionManagement>
    <repository>
      <id>nexus</id>
      <name>Releases</name>
      <url>http://localhost:8081/repository/maven-releases</url>
    </repository>
    <snapshotRepository>
      <id>nexus</id>
      <name>Snapshot</name>
      <url>http://localhost:8081/repository/maven-snapshots</url>
    </snapshotRepository>
  </distributionManagement>

F. The clean and deploy goal in your Java project will build and upload the artifacts to the repository using the server credentials tag from settings.xml

mvn clean deploy -DskipTests

NexusRepository

G. Add a proxy repository

You can add a new proxy repository to your Nexus instance using the following steps

  1. Create a repository from the repositories admin page
  2. Select the maven2 recipe since JBOSS is a maven like repository
  3. Provide a name like “jboss-nexus-repository”
  4. Add this repository to the group you have defaulted your maven to, so that maven can use this as a part of the group it is defaulted to.

NexusProxyRepository

H. Adding your custom jars into the repository

  1. Create a repository with maven2 hosted recipe
  2. Obtain the created repository URL and run the following maven deploy command on your jar file
 mvn deploy:deploy-file 
-Durl=http://localhost:8081/repository/project-customs/ 
-DrepositoryId=nexus -DgroupId=com.oracle 
-DartifactId=ojdbc6 -Dversion=11.2.0.4 
-Dpackaging=jar -Dfile=C:/Users/vishwaka/.m2/ojdbc6.jar 
-DgeneratePom=true
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Maven Stub Project (No POM) 1
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-deploy-plugin:2.7:deploy-file (default-cli) @ standalone-pom ---
Uploading: http://localhost:8081/repository/project-customs/com/oracle/ojdbc6/11.2.0.4/ojdbc6-11.2.0.4.jar
Uploaded: http://localhost:8081/repository/project-customs/com/oracle/ojdbc6/11.2.0.4/ojdbc6-11.2.0.4.jar (1942 KB at 583.2 KB/sec)
Uploading: http://localhost:8081/repository/project-customs/com/oracle/ojdbc6/11.2.0.4/ojdbc6-11.2.0.4.pom
Uploaded: http://localhost:8081/repository/project-customs/com/oracle/ojdbc6/11.2.0.4/ojdbc6-11.2.0.4.pom (392 B at 0.1 KB/sec)
Downloading: http://localhost:8081/repository/project-customs/com/oracle/ojdbc6/maven-metadata.xml
Downloaded: http://localhost:8081/repository/project-customs/com/oracle/ojdbc6/maven-metadata.xml (302 B at 0.2 KB/sec)
Uploading: http://localhost:8081/repository/project-customs/com/oracle/ojdbc6/maven-metadata.xml
Uploaded: http://localhost:8081/repository/project-customs/com/oracle/ojdbc6/maven-metadata.xml (302 B at 0.1 KB/sec)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 15.768 s
[INFO] Finished at: 2017-06-15T11:29:59-07:00
[INFO] Final Memory: 11M/245M
[INFO] ------------------------------------------------------------------------

NexusHostedRepository

You should be able to see this in your repositories assets once the upload is successful. The upload deploy uses credentials from your server.xml configuration so make sure that is available.

3. Upon doing this we need to add the project-custom repository as a member to the maven-public group of repositories

NexusMemberRepository

I. Test by running a clean build of your maven project

  • Delete the folder containing the jar files in the path \.m2\repository\com\oracle\ojdbc6\11.2.0.4
  • Rerun the maven build using mvn clean compile
  • Verify the following logs in the build
Downloading: http://localhost:8081/repository/maven-public/com/oracle/ojdbc6/11.2.0.4/ojdbc6-11.2.0.4.pom
Downloaded: http://localhost:8081/repository/maven-public/com/oracle/ojdbc6/11.2.0.4/ojdbc6-11.2.0.4.pom (392 B at 2.8 KB/sec)
Downloading: http://localhost:8081/repository/maven-public/com/oracle/ojdbc6/11.2.0.4/ojdbc6-11.2.0.4.jar
Downloaded: http://localhost:8081/repository/maven-public/com/oracle/ojdbc6/11.2.0.4/ojdbc6-11.2.0.4.jar (1942 KB at 10846.1 KB/sec)