Monday, March 22, 2010

Text Processing and Data Extraction using sed and awk

Some weeks ago, my friend contacted me for a help. He needs to extract some data from a text file. I am familiar with this stuff, since in my old job I do this quite often. Like usual, the data was just a colon (:) separated file. He needs to extract all fields that starts or contains a particular text. It can appear in different column of the row.

So, I played around using awk, and got something like the following:
awk -F\: '/CSP/ { for(i=1;i<=NF;i++) { if($i ~ "CSP") print $i } }' raw_text_file.txt


Since not long before he asked, I had played around much with regex grouping in python. I just needed to translate the my python regex knowledge in sed. Not much work, of course. So, I came up with the following method, using sed:
sed -r 's/.*(CSP,[0-9]+[:;]).*/\1/g' raw_text_file.txt


Notes: "CSP" is the text that needed to be extracted from the text file, along with some other number(s) after that.

Lots of stuff can be done by regex.
Thanks to Dr.Bunker who taught me so much about regex.

Friday, March 12, 2010

Automatic Display Manager Login using nodm

My posting @ https://answers.launchpad.net/ubuntu/+question/101990

I tried your method and it worked. I have been playing around to get such result for two days. Well, actually there is a little bit of story before I get the result that I wanted. So, I tried some other similar method to Curtis', in my VirtualBox Ubuntu Guest OS. The strange thing is I kept getting problems. The X did not start, and the virtual console resolution got big and too wide.
I had a feeling that I might get different result in "real" system. So, I installed ubuntu 9.10 and applied Curtis' method. It worked. I am glad. What I noticed is that the X is running in virtual terminal #9 (instead of #7). No issue.

I have another method though. During my search, someone mentioned about 'nodm' for automatic display manager. Neat! Yesterday I tried it on VirtualBox with no success. Today I tested it in "real" system and all is well. I really recommend this method.
During 'nodm' installation, I had to remove existing display manager(s) first. Then, set the proper configuration in /etc/default/nodm. Enjoy!