Disable Network Manager and Use Script to Connect to WPA2 Wireless Connection

I would like some of the computers at my work to be able to connect to a specific wireless access point with WPA2 encryption and keep trying to connect (indefinitely) if the Access Point (AP) is not on yet. When the AP is up and running, those computers should be able to connect without any human interaction. There is a possibility that the AP is turned off for a while or restarted, due to some reason, and those computers should be able to reconnect automatically when the AP is back online again.

The default behavior on Ubuntu and Fedora is a little bit different (but far from my need). When a computer is already connected to AP, and the AP suddenly becomes unavailable for a while, there will be a WEP/WPA password dialog. User has to click the connect button again and if the AP is not available yet it will keep showing password dialog. When the user click cancel button, it will stop trying to connect to that particular AP.

Finally I figured out how to set the computer to keep trying to connect indefinitely and no matter how many times the AP becomes unavailable, it will keep doing it until it is (re)connected.

Here is how I did it:
The main issue is with the default behavior of the NetworkManager. I have not found any NetworkManager configuration so I could set it the way I explained earlier. Please let me know if someone knows how to do the exact thing that I need using NetworkManager.

The first step is to disable NetworkManager:
To disable it temporarily, use the following command:
$ sudo service network-manager stop

To permanently disable NetworkManager, do the following:
edit /etc/init/network-manager.conf
comment "start on...", so it would be like the following:
# start on

Next step is to create a wpa supplicant configuration file, do the following:
Create "/etc/wpa_supplicant/wpa_supplicant.conf" file and make it executable.
Put the following configurations inside it:
########### wpa_supplicant.conf ############


ctrl_interface=/var/run/wpa_supplicant

network={
    ssid="your_wlan_essid"
    scan_ssid=1
    key_mgmt=WPA-PSK
    psk="your_wpa2_passphrase"
}


########### end of wpa_supplicant.conf ############

Now, I am not really comfortable to put my plain text wpa2 passphrase inside the config file. So, I use the following method:
$ wpa_passphrase your_wlan_essid your_plain_text_passphrase

Use the output of the command above for your encrypted passphrase

Third step is to edit the /etc/network/interfaces configuration file.
Here is how my config file looks:
############# /etc/network/interfaces ###########

auto lo
iface lo inet loopback

auto wlan0
iface wlan0 inet dhcp
   wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf
   wpa-action /etc/wpa_supplicant/action_wpa.sh

############# end of /etc/network/interfaces ###########

Notice that /etc/wpa_supplicant/wpa_supplicant.conf is the configuration file we just created previously, and /etc/wpa_supplicant/action_wpa.sh should exist from the default ubuntu (11.10) installation.

Now for the test:
Test #1
Make sure your AP is on.
Restart your computer, and after you logged in, you should be connected to AP.

Test #2

Make sure your AP is off.
Restart your computer, and after you logged in, you should not be connected to AP.

Turn on your AP.
Not long after that your computer should be connected to AP.

Test #2
Make sure your AP is on and your computer is connected to AP.
Turn off your AP.
Your computer should be disconnected from AP.
Wait for a while to be sure that your computer is not connected and the retry connection attempt failed.

Turn back your AP on.
Not long after that your computer should be reconnected to AP.


Now, this is exactly what I want for my setting at work.

Distributing Large Files - Implementation Part 1

This post is related to my previous post about distributing large files http://murzal-arsya.blogspot.com/2011/11/distributing-large-files.html.

Finally I implemented my distribution strategy as I explained in my previous post. Let me give a little bit info about the infrastructure.

1 Server:
Brand/Type: IBM System x3650 M3
OS: Centos 6
Service: Samba
NIC: 2 Gigabit

1 Switch: HP Procurve 1810 24g (Gigabit)

10 Clients:
Brand/Type: HP Pavilion HPE
OS: Windows 7
Apps: QGIS
NIC: 1 Gigabit


I have samba configured as shared storage. Within the share folder I put the large image files to be distributed to the clients. My strategy is creating a chain distribution, from the Server to PC-01, PC-01 to PC-02, and so on, until PC-10. I choose robocopy for its simplicity and it does the job well.
So, I created a batch file

Since I need to access the shared folder, I need to make sure that the connection could be established using correct user for authentication. In order to do that, before I run robocopy, I use net use command for authenticating.
>net use \\SERVER\share_images /USER:user1 password1
then I could execute robocopy with something like the following:
>robocopy /E /PURGE /COPY:DAT \\SERVER\share_images C:\images

The most ideal way to trigger the synchronization is when there is change in each share folder, and queuing the sync processes when there are multiple changes detected. It is beyond the scope of this post.

Distributing Large Files

One of the requirements of my new project is to be able to distribute large files to 10-15 clients as fast as possible. Fortunately, all (server and clients) NICs supports Gigabit connectivity.
I tested it first simply by copying large data from server to many clients simultaneously. Of course, it was a disaster. While theoretically the copying speed would decrease proportional to the number of clients, in reality the network connection between clients and server, and also between clients got disrupted. Network was down for sometimes. I did not make any further investigation on what was happening, all I know is this is far from the desired speed even if it was going well.
What's next? I gather some info's on how to distribute large files over LAN. Some suggest bittorrent, parallel filesystem, multicast copy, (win) mesh, and some other way. My concern is that I have to use a solution which is as simple as possible, since there would not be any expert IT guy around in the future.
While bittorrent sounds pretty promising, I had to leave out this option. Not only because of the capability of the human resources that will maintain the system, also it would take some times to prepare the large file to be distributed by bittorrent. Another thing is that small torrent packets could really saturate the network.

About parallel filesystem, I have to ditch this one too, because this would be more complicated and confusing for the basic admin guy. Also I need to mention that all clients are Windows 7, not really sure there are stable parallel filesystem support

Multicast copy should be interesting, but could not find any apps for Windows 7.

So, I figured that I need to make my own simple solution. The goal here is to distribute large files, around 1-5 GigaBytes each, without overloading the connection between source (server) and target (pc).

I simply came up with creating some kind of chain distribution. Transfer file from Server to PC1, and from PC1 to PC2, and so on. Doing this, I could use all of the available incoming gigabit bandwidth as well as the outgoing one. I tested it and worked quite well. Of course there will be a problem if one of the PC is down, then the distribution could not be continued to the rest of the PCs after that. So, rerouting must be done either manually or automatically.

As for the tool to copy the data, one can use simple copy, robocopy, synctoy, and other mechanism. I am planning to use command line robocopy.

I also plan to use load balancing technique using both NICs on the server. So, I can create 2 chains within the network, and perform the transfer simultaneously to those chains.

Text Processing and Data Extraction using sed and awk

Some weeks ago, my friend contacted me for a help. He needs to extract some data from a text file. I am familiar with this stuff, since in my old job I do this quite often. Like usual, the data was just a colon (:) separated file. He needs to extract all fields that starts or contains a particular text. It can appear in different column of the row.

So, I played around using awk, and got something like the following:

awk -F\: '/CSP/ { for(i=1;i<=NF;i++) { if($i ~ "CSP") print $i } }' raw_text_file.txt

Since not long before he asked, I had played around much with regex grouping in python. I just needed to translate the my python regex knowledge in sed. Not much work, of course. So, I came up with the following method, using sed:

sed -r 's/.*(CSP,[0-9]+[:;]).*/\1/g' raw_text_file.txt

Notes: "CSP" is the text that needed to be extracted from the text file, along with some other number(s) after that.

Lots of stuff can be done by regex.
Thanks to Dr.Bunker who taught me so much about regex.

dos to *nix text file

One of annoyances when working in both Windows and Linux OS is the different format of the text files created.
In Linux OS, when you open a text file (using vi) which happened to be created within Windows OS, you will see a lot of "^M" character. Really annoying.

In the past I used dos2unix to convert, but I don't use it anymore since I could just use sed, instead.

Here's the command.

$ sed -i -e 's/\r//g' file_name

To make it even easier, I created a shell script which contain that command, like the following:

#!/bin/bash
sed -i -e 's/\r//g' $*

... and you're ready to go.

Set super+l shortcut key for locking screen in gnome

The default shortcut keys to lock screen in gnome is <ctrl><alt>l (it is small caps 'L'). I do not like that shortcut and prefer to use the combination of "super key" and "l", which actually defined as <Mod4>l. The keybinding GUI setting somehow does not accept any combination of super key with other keys. So, to be able to do that, you need to use gconf-editor.
Perform the following:

run gconf-editor,
option 1, from CLI:

$ gconf-editor

option 2, from GUI:
press <Alt>F2 to get the 'run application' dialog and type in gconf-editor and click 'run'

In gconf-editor window, go to the following key location, by expanding the keys tree:

/apps/gnome_settings_daemon/keybindings

then, double click 'screensaver' key and change the value to: <Mod4>l

and you are done. Test it by pressing super key and l.

There is also feature to search the key name and value, by pressing <Ctrl>f in gconf-editor. Actually, I found the correct location of the 'screensaver' key by searching it.

backup and restore MBR

Partition Table backup

Perform the following command to backup the partition table:

sfdisk -d /dev/hdx > /part/to/pt_info.sfdisk

Partition Table restore

Perform the following command to restore the partition table:

sfdisk -d /dev/hdx < /part/to/pt_info.sfdisk

MBR backup

Perform the following command to backup the MBR:

dd if=/dev/hdx of=/path/to/image count=1 bs=512

MBR restore

Perform the following command to restore the MBR:

dd if=/path/to/image of=/dev/hdx count=1 bs=446

Notice that you need to include the option "count=1 bs=446", otherwise your partition table will be overwritten.

Install chromium browser with flash plugin in ubuntu

Install chromium browser with flash plugin in ubuntu

Add the following repository in your apt's sources.list

deb http://ppa.launchpad.net/chromium-daily/ppa/ubuntu jaunty main

Add the key for it

$ sudo apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 4E5E17B5

Perform the following

$ sudo aptitude update
$ sudo aptitude install chromium-browser

Chromium browser should be installed and ready to be used. If you already installed flashplugin you can add the plugin to chromium-browser. I noticed the browser takes a lot of resource after adding flashplugin, so I no longer user. To set flashplugin perform the following.

$ cd /usr/lib/chromium-browser/plugins
$ sudo ln -s /usr/lib/flashplugin-installer/libflashplayer.so

Auto disable touchpad while typing using syndaemon

Auto disable touchpad while typing using syndaemon

Often time, when working on laptop, my hands touch (tap) the touchpad while typing, which is really annoying, cause I end up writing in different location. Disabling the touchpad permanently is not a good idea, specially if you don't have a mouse with you. Fortunately, there is a very neat tool that monitors keyboard activity and has the ability to disable the touchpad during that event. The tool is syndaemon.

usage example:

$ syndaemon -d -i 1

That would execute syndaemon as daemon and set 1 second as the delay period.

To automatically run this daemon, for example in KDE desktop environment, I created a script which contains the code above, make it executable, and put it in ~/.kde/Autostart folder.

Also check out synclient, taken from the 'man page', synclient - commandline utility to query and modify Synaptics driver options.

"service" like redhat

"service" like redhat

One of many things that I like in Redhat is "service" tool to start, stop registered service (daemons). I realised that in my current ubuntu 8.10, this tool is already installed by default, which is really good. I am pretty sure that it will still available in newer release.
What about in Debian? To have such tool, you need to install "sysvinit-utils" package.

# aptitude install sysvinit-utils

to use it, simply type like the following

# service  start
# service  stop
# service  restart
# service  status

In ubuntu (also in debian, if I'm not mistaken), you can use "tab" command completion, when you are trying to enter the . Very neat!

Trim Filename Extension

Trim Filename Extension

There is built-in functionality in Linux shell to trim extension of a file name.
${filename_env_var%.ext} --> note that dollar ($) sign in the beginning is NOT shell prompt.

Usage example:

$ filename=trace_20090606.txt
$ echo ${filename%.txt}
trace_20090606 --> this is the output of previous command

Broadcast Ping

Broadcast Ping

Perform the following commands:

$ ifconfig eth0 | grep -i bcast
         inet addr:10.10.44.44  Bcast:10.10.255.255  Mask:255.255.0.0

$ ping -b 10.10.255.255

You should see several IP addresses respond to ping.

LiveCD PXE Network Boot

LiveCD Network Boot (pxe boot)

Server Side:
In case you are using VirtualBox, make sure you set the network to internal network.

On Server (Fedora 10):

turn off selinux
method 1: from GUI and reboot
method 2: $ sudo vi /etc/selinux/config
edit --> SELINUX=disabled
save and reboot

turn off firewall
method 1: from GUI (need to click apply button)
method 2: $ sudo iptables -F

configure the IP for NIC (assumption NIC to be configured is eth0):
method 1:use GUI and set the ip to 192.168.0.15 for eth0
method 2:

$ sudo ifconfig eth0 down
$ sudo ifconfig eth0 192.168.0.15 netmask 255.255.255.0 up

mount kubuntu to /mnt/kubuntu
method 1: mount kubuntu_desktop_9.04.iso to /mnt/kubuntu folder
$ sudo mount -o loop kubuntu_desktop_9.04.iso to /mnt/kubuntu
method 2: mount cdrom to /mnt/kubuntu
$ sudo mount /dev/sr0 /mnt/kubuntu

install syslinux:
method 1: from GUI install syslinux package
method 2: $ sudo yum install syslinux

install tftp server:
method 1: from GUI install tftp-server package
method 2: $ sudo yum install tftp-server

add necessary files for network boot in tftp share folder:
$ sudo mkdir -p /var/lib/tftpboot/livecd/pxelinux.cfg
$ sudoe cp -p /usr/lib/syslinux/pxelinux.0 /var/lib/tftpboot/livecd/
$ sudo cp -p /mnt/kubuntu/casper/vmlinuz /var/lib/tftpboot/livecd/
$ sudo cp -p /mnt/kubuntu/casper/initrd.gz /var/lib/tftpboot/livecd/
$ sudo vi /var/lib/tftpboot/livecd/pxelinux.cfg/default
add the following lines to the file:
############## start of pxe default ##############
default kubuntu
timeout 30

label kubuntu
kernel vmlinuz
append boot=casper netboot=nfs nfsroot=192.168.0.15:/mnt/kubuntu initrd=initrd.gz --
############## end of pxe default ##############

activate tftp server:
$ sudo chkconfig tftp on

restart xinetd:
method 1: from GUI
method 2: $ sudo service xinetd restart

install dhcp server:
method 1: from GUI install dhcp package
method 2: $ sudo yum install dhcp

configure dhcp server:
$ sudo vi /etc/dhcpd.conf

replace the content with:
############## start of dhcpd.conf ##############
allow booting;
allow bootp;
ddns-update-style interim;
ignore client-updates;

subnet 192.168.0.0 netmask 255.255.255.0 {
option subnet-mask 255.255.255.0;
option broadcast-address 192.168.0.255;
range dynamic-bootp 192.168.0.200 192.168.0.240;
next-server 192.168.0.15;
filename "livecd/pxelinux.0";
}
############## end of dhcpd.conf ##############

restart dhcp server:
method 1: from GUI
method 2: $ sudo service dhcpd restart
method 3: if you have more than one NICs, and you want to use dhcp only for eth1 perform the following,
$ sudo dhcpd -d eth1

install nfs server:
method 1: from GUI install nfs-utils package
method 2: $ sudo yum install nfs-utils

configure nfs server:
$ method vi /etc/exports
add the following line:
############## start of /etc/exports ##############
mnt/kubuntu 192.168.0.0/255.255.255.0(async,no_root_squash,no_subtree_check,ro)
############## end of /etc/exports ##############

restart nfs server:
method 1: from GUI
method 2: $ sudo service nfs restart

check nfs export:
$ sudo showmount -e 192.168.0.15


Client Side:
In case you are using VirtualBox, make sure you set the network to internal network.
Set BIOS boot option to PXE Network Boot
Boot the computer

You should be able to see the client PC getting DCHP and loading the LiveCD from network.

yesterday's date in linux

To get yesterday's date in linux, simply use the following command:

$ date --date='1 day ago'

vnc over ssh

For those who like to use vnc to work with remote computer, you should be aware that the traffic is not encrypted. Please refer to tightvnc faq page for information about this. Just like what recommended in the faq, ssh is required to secure the traffic.

Create a tunnel between your local and remote computer, and one way to do this is:
$ ssh -CNT username@remote_computer -L 5901:127.0.0.1:5900

Now, simply set your vnc viewer to access localhost with port 5901.

An example of using vncviewer is:
$ vncviewer localhost:5901

ecryptfs == privacy

ecryptfs == privacy

Simple and nice way to protect your documents in linux.
Install ecryptfs by executing the following command:
$ sudo apt-get install ecryptfs-utils

Create a directory where you want to put those private documents. For example I will create a folder in my home, named "Personal", and set the permission only to owner.
$ mkdir ~/Personal
$ chmod 700 ~/Personal

Mount that directory as eCryptfs filesystem.

$ sudo mount -t ecryptfs ~/Personal ~/Personal
I chose passphrase, aes, and 16-bit. For plaintext passthrough I chose "no" (no non-ecrypted files allowed in the mount).

Move, copy, create all private documents into ~/Personal directory.

When done, umount ~/Personal by executing:
$ sudo umount ~/Personal

All the files inside that directory can not be opened unless you mount it as ecryptfs filesystem.

Next time you mount, use the following command, so it will only ask for passphrase.
$ sudo mount -t ecryptfs ~/Personal ~/Personal -o key=passphrase,ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_passthrough=n

ssh tunneling

ssh tunneling

In the client execute:

$ ssh -N -D user@CompB

where CompB is the server

To use this ssh tunneling with internet browser, for example firefox, change the "Connection Settings", under "Manual proxy configuration", update the "SOCKS Host:" to localhost:any_free_port. Notice that any_free_port here must be the same one as the one used in ssh.

Opera browser does not have socks proxy configuration. One way to do it is by using "tsocks."
Install tsocks:

$ sudo apt-get install tsocks

edit /etc/tsocks.conf, and change
server = 127.0.0.1 # this is localhost
server_port = # again, this must match with the port used in ssh.

screen

I work remotely a lot, and mostly using ssh, since it's more secure than telnet. There's a time when the process takes very long time, and I don't want to always be connected the whole time.
The problem is, when I put the process in the background (using ctrl-z and bg, or using "&" when executing), when I exited the remote shell, all processes belong to that remote shell will be terminated.

One way to resolve this issue is using "nohup". This is very useful when the process does not require any interaction. Still I'm not satisfied with "nohup."

Then I came across this very neat tool, "screen." Very elegant way to do things remotely.

Do the following:

$ screen

it will give you the session, and start to do any activity. So, when I want to leave the remote shell. Press --> ctrl-a, d. That command will detach the screen session, and it's save to exit the remote shell.

To resume that particular session. Login again to the remote computer, then look at the detached session, by:

$ screen -ls
$ screen -r <session_id>

rsync + ssh => resumable download

rsync + ssh => resumable download

Sometimes ago, I felt the necessity of being able to resume my download from another computer, specially if the file is quite big. In this case, it's time to use rsync. Since mostly I use ssh to access remote computer, so I combine rsync with ssh.

Here's how to do it:

$ rsync --partial --progress --rsh=ssh user@server:/serverpath/to/file /localpath/to/file

Now, what if you need to use non-standard ssh port? Let's say you need to connect to ssh via port 7890. The command will be like the following:

$ rsync --partial --progress --rsh='ssh -p7890' user@server:/serverpath/to/file /localpath/to/file

The neat thing is that there is also progress indicator shown.

Resolve Windows hostname in Linux

Resolve Windows hostname in Linux

I work a lot in heterogeneous environment, and ofcourse, windows is one of the system in that network. So, I would like to be able to resolve windows hostname (netbios) and access it's resource. In order to do that, there are packages needed to be installed first.

$ sudo apt-get install winbind

after that, edit /etc/nsswitch.conf configuration file.
Search for word "hosts", and add "wins" at the end of the line. Below is an example of "hosts" configuration after update.

# /etc/nsswitch.conf
#
# ....

hosts: files mdns4_minimal [NOTFOUND=return] dns mdns4 wins

#....

after that, restart winbind service (if necessary). Now, windows computers should be able to be accessed using it's windows hostname. Pretty convenient way to access those windows computer in the network.

If you are still not able to resolve windows hostname, perhaps there is a firewall running in your computer. In that case you need to add extra rule to allow udp.

I use firestarter for my personal firewall, and what I did was edit the /etc/firestarter/inbound/setup configuration file.
Search for:
$IPT -A INBOUND -p udp -m state --state ESTABLISHED,RELATED -j ACCEPT
and put the following rule right after it:
$IPT -A INBOUND -s $NET -p udp -m state --state NEW -j ACCEPT

restart firestarter, to load the new config. Try to access windows computer using it's hostname again.


I just found out a way to configure UFW for this matter.
First edit: /etc/default/ufw
Search for:

IPT_MODULES="nf_conntrack_ftp nf_nat_ftp nf_conntrack_irc nf_nat_irc"

and change it to:
IPT_MODULES="nf_conntrack_ftp nf_nat_ftp nf_conntrack_irc nf_nat_irc nf_conntrack_netbios_ns"


Then: reload ufw
# sudo ufw reload

Next step: Configure UFW Rules
Allow outgoing UDP to port: 137, 138
Allow outgoing TCP to port: 139, 445