Posts

Showing posts from 2016

[Haar Classifier] Train your own OpenCV Haar classifier

I just keep a record for myself because there are a lot of documents teaching how to train your haar classifier and almost of them seem to don't work well. The following 2 items are clear and easy to understand.

The Data Image Source (cars) I use.
http://cogcomp.cs.illinois.edu/Data/Car/

1. Train your own OpenCV Haar classifier
https://github.com/mrnugget/opencv-haar-classifier-training

find ./positive_images -iname "*.pgm" > positives.txt
find ./negative_images -iname "*.pgm" > negatives.txt

perl bin/createsamples.pl positives.txt negatives.txt samples 550\
  "opencv_createsamples -bgcolor 0 -bgthresh 0 -maxxangle 1.1\
  -maxyangle 1.1 maxzangle 0.5 -maxidev 40 -w 48 -h 24"

python ./tools/mergevec.py -v samples/ -o samples.vec

opencv_traincascade -data classifier -vec samples.vec -bg negatives.txt\
  -numStages 10 -minHitRate 0.999 -maxFalseAlarmRate 0.5 -numPos 1000\
  -numNeg 600 -w 48 -h 24 -mode ALL -precalcValBufSize 1024\
  -precalcIdxBufSi…

[Image] How to resize, convert & modify images from the Linux

Installation
$ sudo apt-get install imagemagick

Converting Between Formats
$ convert howtogeek.png howtogeek.jpg

You can also specify a compression level for JPEG images:
$ convert howtogeek.png -quality 95 howtogeek.jpg

Resizing Images
$ convert example.png -resize 200×100 example.png
  - to force the image to become a specific size – even if it messes up the aspect ratio
$ convert example.png -resize 200×100! example.png

$ convert example.png -resize 200 example.png
$ convert example.png -resize x100 example.png

Rotating an Image
convert howtogeek.jpg -rotate 90 howtogeek-rotated.jpg

Applying Effects
ImageMagick can apply a variety of effects to an image.
  - For example,   the following command applies the “charcoal” effect to an image:
$ convert howtogeek.jpg -charcoal 2 howtogeek-charcoal.jpg
  - the “Implode” effect with a strength of 1:
# convert howtogeek.jpg -implode 1 howtogeek-imploded.jpg

Batch Processing
for file in *.png; do convert $file -rotate 90 rotated-$file; done



R…

[TensorFlow] My case to install TensorFlow with GPU enabled

My Operation System is Ubuntu 14.04 LTS 5 and GPU card is GeForce GTX 750Ti

1. Go to nvidia.com and download the driver (NVIDIA-Linux-x86_64-367.44.sh) 2. For Nvidia to find linux header files (*):
$ sudo apt-get install build-essential linux-headers-$(uname -r)

3. To enable full screen text mode (nomodeset):
$ sudo gedit /etc/default/grub
>> Edit GRUB_CMDLINE_LINUX_DEFAULT="quiet splash nomodeset"
Save it and reboot
$ sudo update-grub
$ sudo reboot

4. Log into with Ctl +Alt + F1

5. Stop the X Server service
$ sudo service lightdm stop

6. Install nVidia driver
$ sudo ./NVIDIA-Linux-x86_64-367.44.sh
7. Install CUDA (GPUs on Linux) Download and install Cuda Toolkit https://developer.nvidia.com/cuda-downloads sudo dpkg -i cuda-repo-ubuntu1404-8-0-local_8.0.44-1_amd64.deb sudo apt-get update sudo apt-get install cuda
8. Download and install cuDNN https://developer.nvidia.com/cudnn tar xvzf cudnn-8.0-linux-x64-v5.1.tgz cd cuda
sudo cp include/cudnn.h /usr/local/cuda-8.0/includ…

[OpenCV] To install OpenCV on Debian and create a test project using Netbeans

Image
This document is the steps to install OpenCV on Debian and create a test project using Netbeans for my reference in case.

#Prepare the build environment GCC、Cmake、pkgconfig
$sudo apt-get -y install build-essential cmake pkg-config
#Install ImageI/O libraries
$sudo apt-get -y install libjpeg62-dev libtiff4-dev libjasper-dev
#Install Viode I/O libraries
$sudo apt-get -y install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev $sudo apt-get -y install libdc1394-22-dev libxine2-dev libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev
#Install GTK+2.x and QT libraries
$sudo apt-get -y install libgtk2.0-dev libqt4-dev
#(Optional)Install support for Firewire video cameras
$sudo apt-get -y install libdc1394-22-dev
#(Optional)Install video streaming libraries
$sudo apt-get -y install libxine-dev libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev
#(Optional)Install the Python development environment and the Python Numerical libraries
$sudo apt-get -y install python-dev python-numpy
#(Op…

[Hadoop] Setting up a Single Node Cluster

Basically these resource links are good enough to do a single node cluster of Hadoop MapReduce. But I still want to add some comments for my reference.
http://www.thebigdata.cn/Hadoop/15184.html
http://www.powerxing.com/install-hadoop/

Login the user "hadoop"
# sudo su - hadoop
Go to the location of Hadoop
# /usr/local/hadoop
Add the variables in ~/.bashrc
export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64 export HADOOP_HOME=/usr/local/hadoop export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop export HADOOP_INSTALL=/usr/local/hadoop export PATH=$PATH:$HADOOP_INSTALL/bin export PATH=$PATH:$HADOOP_INSTALL/sbin export HADOOP_MAPRED_HOME=$HADOOP_INSTALL export HADOOP_COMMON_HOME=$HADOOP_INSTALL export HADOOP_HDFS_HOME=$HADOOP_INSTALL export YARN_HOME=$HADOOP_INSTALL
Modify $JAVA_HOME in etc/hadoop/hadoop-env.sh
export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64
Start dfs and yarn
# sbin/start-dfs.sh# sbin/start-yarn.sh
Finally, we can try the Hadoop MapReduce example as follows:
# bin/hadoop…

[Tensorflow] Fizz-Buzz example enhancement

Image
I am just based on this Fizz-Buzz example as below to add 2nd convolution layer and guess what? The result is quicker to be learn. But, this is just the first step to learn "Deep Learning"...
There is still a lot of things and knowledge that need to learn more.
http://joelgrus.com/2016/05/23/fizz-buzz-in-tensorflow/

Before



After


Reference
http://www.slideshare.net/WrangleConf/wrangle-2016-lightning-talk-fizzbuzz-in-tensorflow

[Neutron] The first glance of L3HA mode in OpenStack Neutron ( Liberty version )

Image
I just quickly take the first glance of L3HA mode in OpenStack Neutron ( Liberty version ) and is based on my tenant environment as follows:
My tenant environment # neutron router-list




# neutron net-list
# neutron subnet-list
The Topology view looks like this:

Here I have 2 instances in my tenant:


So, if I use the instance: daanny_vm1 to ping danny_vm2, due to the different subnets, this action will trigger L3 vrouter function.

# ping 192.168.66.4 ( danny_vm2 )

# ip netns exec qrouter-f1e03fef-cccf-43de-9d35-56d11d636765 tcpdump -eln -i qr-4433f31f-5d icmp


The interface qr-4433f31f-5d is my subnet 192.168.44.0/24's gateway port as follows:
# neutron --os-tenant-name danny port-list | grep 4433f31f-5d | 4433f31f-5d93-4fe4-868a-04ddcc38be20 | | fa:16:3e:25:22:b3 | {"subnet_id": "d169f180-4304-42f0-b11f-e094287bcd00", "ip_address": "192.168.44.1"} | Keepalived related L3HA mode is havily relied on…

[Ceilometer] To survey how to improve the performance of OpenStack Ceilometer

Image
Frankly speaking, OpenStack Ceilometer will suffer some kind of performance issues sooner or later if you don't modify or tune the configuration. The issues has two parts that need you to consider. One is the message bus and API loading, and the other is database. However, I find some best practices which are easy and quick for us to adopt. Here you go:
1. Telemetry(Ceilometer) best practiceshttp://docs.openstack.org/admin-guide/telemetry-best-practices.html a. Data collectionBased on your needs, you can edit the pipeline.yaml configuration file to include a selected number of meters while disregarding the rest. By default, Telemetry service polls the service APIs every 10 minutes. You can change the polling interval on a per meter basis by editing the pipeline.yaml configuration file. for example: vim /etc/ceilometer/ceilometer.conf => evaluation_interval=120 vim /etc/ceilometer/pipeline.yaml => interval: 120

you can delay or adjust polling requests by enabling the jitter su…

[Linux] Why does Linux require moving IP from eth interface to bridge interface?

This could be a common problem if you have KVM ( or other hypervisor in Linux ) on your physical server and want to use bridge mode with your VMs. At the same time, you also want to let your physical server has the network that can be accessed from other hosts at the same network subnet. At this moment, when a network interface (e.g., eth0) is added to a Linux bridge (e.g., br0), the IP address must be removed from eth0 and added to br0 for the networking to function properly.

I find some answers as follows:
http://unix.stackexchange.com/questions/86056/why-does-linux-require-moving-ip-from-eth-interface-to-bridge-interface
http://unix.stackexchange.com/questions/52674/why-doesnt-eth0-get-an-ip-address-when-set-up-in-a-bridge

==>
The NIC represents the uplink cable. A cable is layer 1, not layer 3. Now the Bridge works as the device that is being addressed for network traffic (incoming) on the server - either on layer 2 (Ethernet/MAC) and/or layer 3 (IP). So the device that responds to …

[Neutron] Neutron Cheat Cheat Sheet

Image
In recent days, the hand drawing style is becoming more and more pervasive in Taiwan. I don't know how it happens, but at least I can draw the Neutron cheat cheat sheet to echo this kind of style for fun.

P.S: This picture is originally for my colleagues to trouble shooting the Neutron networking problems.

[Neutron and SDN] Warm-up for understaning the integration of Neutron and SDN

Image
I just spend some time to study the integration of Neutron and SDN. Furthermore, I also take a look at how ODL and ONOS are integrated with OpenStack Neutron. The following content and picture ( with URL link) are excerpted from the variety of resource in the internet. Some of sections has my comment with P.S. I think it can give you a clear concept of Neutron and SDN controller.

Neutron and SDN
P.S: This picture gives an overall architecture about Neutron and SDN controller that are integrated together. http://thenewstack.io/sdn-controllers-and-openstack-part1/

When an OpenStack user performs any networking related operation (create/update/delete/read on network, subnet and port resources) the typical flow would be as follows: The user operation on the OpenStack dashboard (Horizon) will be translated into a corresponding networking API and sent to the Neutron server. The Neutron server receives the request and passes the same to the configured plugin (assume ML2 is configured with an ODL m…

[Proxy ARP] What is Proxy ARP?

Image
Why I mention Proxy ARP is because in OpenStack environment, this function exists in FIP namespace with DVR enabled. The way to check whether the FIP namespace's Proxy ARP enabled is here:
# ip netns exec fip-545b57e2-0fa5-46da-89a2-591f7a5474ce cat /proc/sys/net/ipv4/conf/fg-2f3f4992-23/proxy_arp
1

We also can see the arp and ip mapping:
# ip netns exec fip-545b57e2-0fa5-46da-89a2-591f7a5474ce ip neighbor
10.12.20.32 dev fpr-6ddabb95-1 lladdr 42:72:88:ca:07:72 STALE

The following is the explaination about Proxy ARP
http://linux-ip.net/html/ether-arp-proxy.html

Occasionally, an IP network must be split into separate segments. Proxy ARP can be used for increased control over packets exchanged between two hosts or to limit exposure between two hosts in a single IP network. The technique of proxy ARP is commonly used to interpose a device with higher layer functionality between two other hosts. From a practical standpoint, there is little difference between the functions of a packet-fi…

[Docker] the first experience with building docker image

Image
It is essential to look for the docker image first if you need some services or functions running on docker container. But, once you want to customize it, you probably need to build your own docker image. The official document gives you a very complete description for you to do so. Please refer to this https://docs.docker.com/engine/userguide/containers/dockerimages/
The following command list are my steps to build a customized Drupal docker image.
The are two ways to build your own image:
1. Updating and committing an image First, it would be better to have a Docker Hub account like this:


Second, to create a repository for your docker image.


If it's done, you can see this:



So, we can continue to the next step.
# Download the offical Drupal docker image
$ docker search drupal
$ docker pull drupal
$ docker images

# Create a container and update it ( be aware of the follwing parameters )
$ docker run -i -t --name danny_drupal -p 8000:80 drupal /bin/bash
  -i, --interactive           …

[Python] Problem with Python logging RotatingFileHandler in Django website

If seeing the log files are not rotated properly or correctly in Django web site, you most likely encounter the problem as the following article described:

Problem with Python logging RotatingFileHandler in Django website
"The log is done via RotatingFileHandler which is configured with 10 log files, 1000000 byte each. The log system works, but this are the log files I get: -rw-r--r--1 apache apache 83Jul2313:30 hr.log -rw-r--r--1 apache apache 446276Jul2313:03 hr.log.1-rw-r--r--1 apache apache 999910Jul2306:00 hr.log.10-rw-r--r--1 apache apache 415Jul2316:24 hr.log.2