Tag Archives: shell script

sed a powerful command line editor.

Recently I had to work with a lot of xml files in connection to my work. Now there are few 10’s of them less 100 arranged in a several directories. The original xml files that I had was passed on to me by a colleague who obviously had a different directory structure and she developed it accordingly for her directory layout. Also there is a special xml tag which lets the file identify itself in terms of its absolute path. Something like <path file=”/l1calo/segments/dqm/<thisfile>.xml” /> Now I need to repace the root path “/l1calo” with “myhome/l1calo”. Only then could I load these xml files to the application. Otherwise it wouldn’t work. So How do I do it. I could open each file in gvim and have done a pattern substitution. But this is extremely inefficient, given the fact that I have to go through 10 directories which on an average contains 5 or 6 xml files.

This problem can be easily solved using sed. A command line text editor. I just had to loop through all the directories and execute sed to replace the specific pattern with my choice. Here is command .  cd to the root directory of your xml’s and then do

for dir in `ls -d */`; do (cd $dir; sed -i 's/\/l1calo\/segments/\/home/\/myhome\/l1calo\/segments/g' * ); done;

Thats it.


Leave a comment

Filed under Unix

SSH to remote machine by providing password from a file

My work involves frequent login to remote machines. And in most cases I have to use multiple terminals to do multitasking. Owing to the high security demands of CERN computers the passwords has to be too long and complicated. Its very inconvenient to type your password each and every time you login. Here is a shellscript which helps you to login to the system just by typing one command.

For the script to work, there are some prerequisites:

1) You should install the package  sshpass
Its easy to install it in ubuntu:  

sudo apt-get install sshpass

For other distributions, you can download the source(http://sourceforge.net/projects/sshpass/)  and compile it

2) You have to type your password in file (My location is /usr/local/bin/cernpwd).

Make sure that the file permission is set to 000. (chmod 000 /usr/local/bin/cernpwd)

3) Here is the script (name sshcern)

#! /bin/bash
# This is a shell script to remote login (ssh) by giving password through script
#Author: Thomas Mathew tm1729@gmail.com
sudo cat /usr/local/bin/cernpwd > ~/.tmp
sshpass -f ~/.tmp ssh -Y user@lxplus.cern.ch && rm ~/.tmp 
#After this you can change /etc/sudoers file by typing sudo visudo  to the following so that u dont have to type sudo password for every shell #Defaults env_reset,!tty_tickets, timestamp_timeout=10

Now copy the script to the folder /usr/local/bin and make it executive.

Now the only thing you need to do is type sshcern 😉

Drawback: The drawback with  this code is that as long the shell script is running ~/.tmp file will not be deleted. So it is accessible while the script is running. Improvements and suggestions are expected

Comment: As of now, the code will not work in Ubuntu 11.04 owing to this bug https://bugs.launchpad.net/ubuntu/+source/sshpass/+bug/774882

An alternative, more secure method of achieving the same is by copying the ssh keys to the remote computer. More inf o is available at http://www.slac.stanford.edu/exp/atlas/computing/ssh.html

Update 1: By Rotiyan

Login to lxplus, manually type in your password.
remember to do ssh -Y for x forwarding.. so that you have an x session on the host machine.
Then start a screen session..
Just type screen and you will see that a new shell has started in the same terminal
. Now type zsh -l or bash -l whichever is your favorite shell

Then do gnome-terminal &
This will give you a new terminal. This way you can make n number of shells.

ssh -Y: As mentioned earlier.. this will give you an xsesison on the host machine.. So that you can open programs like kdevelop, gvim and all of them in their respective windows.

screen: screen is a wonderful tool which comes really handy for people like us. It doesn’t end your session at the host machine even if your ssh connection is lost due to some technical reasons. Another advantage is that, if you want to run a program overnight on the remove server (quite unlikely thanks to grid.. but still), you can start the program and even if you shutdown the client computer the program will be executed for you in the host machine. http://www.linuxjournal.com/article/6340.

Why do we start a screen session ?
This is to make sure that our daughter gnome-terminals are not killed if we accidentally close the mother terminal.

I think this work flow is much secure. But I agree that this is not as easy as sshcern.

Update 2: By Rotiyan

This is also an alternative http://www.thegeekstuff.com/2008/06/perform-ssh-and-scp-without-entering-password-on-openssh/

There are two aspects of this hack I want to discuss.

1) Our objective is to have a password less login. So just hit enter while ssh-keygen ask you a pass phrase so that you don’t have to type in a pass phrase for the login. I am not sure how “safe” is this (need to talk to experts) but at the moment I think its safe because the login session is authenticated via a dual key cryptographic technique (public-private key match).

2) Another point is that If you give a pass phrase while you do ssh-keygen, you can start an ssh session by entering ssh pass phrase. This is more secure than ‘passworded’ ssh session. You don’t have to worry about some one eves dropping your password. If some one gets access to your private key, s/he might be able to login to the remote machine. But can’t do anything more (like accessing your email accounts and stuff)

Well now that being said.. I must admit that my attempt to do this was a failure. Firstly because my login name at host and remote machines are different. Further I have some domain name issues as well. Also I am behind a firewall (this might complicate things)

I would like to if this worked for someone. I will give an update to this If at all I am successful

Update 3: Rotiyan

start a screen session in your host machine
log into the remote machine

once you are done with your activities on the remote machine just detach the screen session.
now whenever you want to login into the remote machine just attach the screen session.

Now this works even if you have a connection interruption. You can retrieve your remote session even after you have suspended your host machine.

Leave a comment

Filed under ATLAS, CMS, Unix