40m QIL Cryo_Lab CTN SUS_Lab TCS_Lab OMC_Lab CRIME_Lab FEA ENG_Labs OptContFac Mariner WBEEShop
  40m Log  Not logged in ELOG logo
Entry  Thu Jul 9 16:39:03 2015, Koji, Configuration, General, How to set up your own summary page environment on the LDG cluster 
    Reply  Mon Jul 13 18:27:27 2015, Eve, Configuration, General, How to set up your own summary page environment on the LDG cluster 
Message ID: 11399     Entry time: Thu Jul 9 16:39:03 2015     Reply to this: 11405
Author: Koji 
Type: Configuration 
Category: General 
Subject: How to set up your own summary page environment on the LDG cluster 

Here is the summary of my investigation how to set up my own "summary page" environment on the LDG (LIGO Data Grid) cluster.

Here all albert.einstein must be replaced with your own LIGO.ORG user name.


1. Obtain LDAS cluster account

Run the following from any of the terminal and use your LIGO.ORG credential
ssh albert.einstein@ssh.ligo.org
You will be suggested to visit a particular web site. Fill the form on the web site and wait for the approval e-mails.

2. Use LDG SSH login portal

Once you received the approval of the account, you should be able to log in to the system. Type the following command again from your local terminal
ssh albert.einstein@ssh.ligo.org
You are asked to select the site and machines. Select 2- CIT and b. ldas-pcdev1, c. ldas-pcdev2, or d. ldas-pcdev3.

3. Setup bash environment

Setting up the python library path is very important for the proper processing.

Here is my setup for .bash_profile

# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
    . ~/.bashrc
fi

if [ -f ~/.profile ]; then
        . ~/.profile
fi

# User specific environment and startup programs
PATH=$PATH:$HOME/bin
export PATH

# So that ssh will work, take care with X logins - see .xsession
[[ -z $SSH_AGENT_PID && -z $DISPLAY ]] &&
  exec -l ssh-agent $SHELL -c "bash --login"

and .bashrc

# .bashrc

# Source global definitions
if [ -f /etc/bashrc ]; then
    . /etc/bashrc
fi

# Set Python environment (based on gpwy-env script)

# clean path environment variable of duplicate entries
cleanpath() {
    if [[ -z "$1" ]]; then
       $1=PATH
    fi
    # map to local variable
    local badpath=$(eval echo \$$1)
    badpath=${badpath%%:}
    # remove duplicates
    badpath="$(echo "${badpath}" | awk -v RS=':' -v ORS=":" '!a[$1]++')"
    # remove trailing colon
    badpath=${badpath%%:}
    # reset variable and export
    eval $1=${badpath}
    eval export $1
}

# set PATH
cleanpath PATH
cleanpath PYTHONPATH

PATH=${HOME}/.local/bin:${PATH}
PYTHONPATH=${HOME}/.local/lib/python2.6/site-packages:${PYTHONPATH}

The order of cleanpath and PYTHONPATH= is important as we want to use the local library installation before anything kicks in.

4. Install required Python libraries

Run the following lines with this order so that they are installed in your "~/local"

# PIP installation
wget https://bootstrap.pypa.io/get-pip.py
python get-pip.py --user
# numpy, scipy, distribute, matplotlib, astropy, importlib installation
pip install numpy --upgrade --user
pip install scipy --upgrade --user
pip install distribute --upgrade --user
pip install matplotlib --user --upgrade
pip install astropy --upgrade --user
pip install importlib --user --upgrade

# We need to use dev branch of gwpy to run gwsumm propery
cp -r /home/detchar/opt/gwpysoft/lib/python2.6/site-packages/gwpy* ~/.local/lib/python2.6/site-packages/

# gwsumm installation
pip install --user git+https://github.com/gwpy/gwsumm

5. Setup summary pages for the 40m

Copy summary page setting from Max's directory.

cp -r ~max.isi/summary ~/

And make temporary directory for the summary pages.

mkdir /usr1/albert.einstein/summary

6. Modify typos in gw_summary_custon

Use your own editor to fix typos

emacs ~/summary/bin/gw_daily_summary_custom

replace max.isi to albert.einstein
change summary40m -> summary


Now the installation is done. From here, the description is for the routine procedure.

7. Run your summary page code

Run Kerberos authentication
kinit albert.einstein@LIGO.ORG

Run a summary page code for a specific date (e.g. for Jul 1st, 2015)
bash ${HOME}/summary/bin/gw_daily_summary_custom --day 20150701

The result can be checked under

https://ldas-jobs.ligo.caltech.edu/~albert.einstein/summary/
https://ldas-jobs.ligo.caltech.edu/~albert.einstein/summary/day/20150701/

Rerun a code for a specific page. This requires the page structure already exists.
The red texts should be modified depending on what ini file you want to run for what day.

/home/albert.einstein/.local/bin/gw_summary day --on-segdb-error warn --verbose  --output-dir . --multi-process 20 --no-html  --ifo C1 --archive C1EVE 20150630  --config-file /mnt/qfs2/albert.einstein/public_html/summary/etc/defaults.ini,/mnt/qfs2/albert.einstein/public_html/summary/etc/c1eve.ini

This command can actually be found in
https://ldas-jobs.ligo.caltech.edu/~albert.einstein/summary/gw_summary_pipe.sh

8. Some useful command

To check which python library is used
python -c 'import gwpy; print gwpy.__file__'

To list installed python libraries and versions
pip list

This should return the list like the following.

...
astropy (1.0.3)
...
gwpy (0.1b1.dev121)
gwsumm (0.0.0.dev854)
...
matplotlib (1.4.3)
...
numpy (1.9.2)
...
scipy (0.15.1)
...

 

 

ELOG V3.1.3-