Tag Archive

amateur astronomy awk bash b[e] supergiant cartoon conference convert evolved star exoplanet fedora figaro fits fun galaxy iraf large magellanic cloud latex linux lmc machine learning magellanic clouds massive star matplotlib meteor mypaper paper peblo photometry planet pro-am pyraf python red supergiant scisoft skinakas observatory small magellanic cloud smc spectroscopy starlink talk ubuntu university of crete video x-ray yellow hypergiant

Merging catalogs and creating unique identifier in bash

For a certain project I had created a number of photometric catalogs, each one corresponding to a specific observing field. I would like to construct the final (merged) one but for this I needed to add a unique source identifier at the beginning of each row. I decided to create a F#-**** tag for each source with “F#” corresponding to the field id and **** to a counter for each source per field. The final command was:

for i in {1,2,4,5,6,7,8,9,10,11,12,13,16};do echo F$i.matches.all.cat;awk -v id="$i" 'FNR>1 {print "F"id"-"1+c++, $0}' F$i.matches.all.cat >> results.tmp; done

So the command reads all the specific numbers for which a catalog with a filename of F*.matches.all.cat exists. The number of each field ($i) is parsed as an external variable (id) to awk which places it as the unique identifier “Fid-counter” with the incremental “counter” (1+c++) corresponding actually to the number of row (1+counter to begin from 1 instead of 0 – FNR avoids the first line of each catalog which is a column description). All results are written appended to the output file results.tmp (created automatically when non-existing).

Then, we can use sed to add the header:

sed -i '1i\#SourceID ...' results.tmp

Bash scripting: convert float to integer

As bash is not able to handle floats the best way is to convert a float number (which is represented as a string, for example an output with awk) into an integer.

To do this [1,2]:

[1] Stackexchange.com/Unix & Linux
[2] Linuxquestions.org

Multiple-line comment for shell scripts

How to comment a block of text without inserting the ‘#’ character to all lines?
We just use a ‘<<COMMENT_STRING’ (COMMENT_STRING can be obviously whatever we want) before the line of text we want to comment out and ‘COMMENT_STRING’ after the last line.

An example:

This is a test comment - lol!


ps. In source 1 the “:” is placed in front of the “<<COMMENT_STRING”, which is not mentioned in source 2 and from what I tested with bash scripts it seems not necessary.

Zip multiple directories separately

In order to zip simultaneously a number of directories, but keep them separate, the following command will just do the job (in bash):

for i in *; do zip -r $i.zip $i; done

(caution: all directories inside which this command is executed will be zipped, so make a separate directory only to create the zip files)


Bash commands to combine files

Suppose that you want to combine two (or more) files, containing for example a series of data. There are, actually, two ways to do it in bash:

$ paste temp1 temp2 > temp
which adds the contents of the two files horizontally, like columns, and

$ cat temp1 temp2 > temp
which adds the contents vertically, one after the other.

Simple as that.

Common commands of Scisoft and Starlink

Scisoft includes a number of Starlink programs but not all. So perhaps someone (in this case me!) needs to run something that it is not included in the Scisoft (like DIPSO). So, nothing wrong to go on and install both. But in this case that someone has to be careful not to mix the commands.

As the Starlink programs are not available from the beginning within the setup script of Scisoft, some initial work has to be done [how to do it?]. If an alias is selected then its names for the environment and the program has to be different than what Starlink uses. In the example of setting up FIGARO through Scisoft we used “FIG_DIR” and “figaro” as the alias in the .bashrc, which conflicts with “figaro” of Starlink. The best way is to name them differently (like “SCIFIG_DIR” and “scifigaro”). By that we have available both programs, either the Starlink version by typing “figaro” or the Scisoft version by typing “scifigaro” (may be different versions depending on the Scisoft/Starlink releases).

Installing Starlink

Installing and running Starlink is pretty much straightforward. Firstly, get the latest release (Namaka, when this post was written) and extract the file wherever you want. Then select which shell you want to use and follow the post-installation run-up or the info given below (supposing it was downloaded and extracted in /home/user/soft/star-namaka/).

Bash case:

Open the .bashrc file (found at the /home/user/ directory) and add these lines:

export STARLINK_DIR=/home/user/soft/star-namaka/
. $STARLINK_DIR/etc/profile

In this case the Starlink programs are available with the start of the bash shell.

If you don’t want this to happen (don’t ask me why…) then you should define an alias, so type instead:

export STARLINK_DIR=/home/user/soft/star-namaka/
alias star='. $STARLINK_DIR/etc/profile'

Now, in order to start Starlink after the start of the shell type “star” (or whatever you have chosen to name it).

Tcsh case:

The case of a tcsh shell (or csh) is more or less similar by defining the environment and sourcing the necessary files ( login and cshrc under star-namac/etc/). Open the .tcshrc (or .cshrc, at the home directory as previously)  and type:

setenv STARLINK_DIR /home/user/soft/star-namaka/
source $STARLINK_DIR/etc/login
source $STARLINK_DIR/etc/cshrc

So, starting a tcsh shell will automatically initialize the Starlink programs.

Happy processing !!

Figaro within Scisoft

Scisoft (7.5) includes also a number of shared programs available to use. But they are not immediately available when invoking Scisoft, like FIGARO, one of the programs provided with STARLINK. There are two ways to start FIGARO (both provided by Giannis K. at Computer Center support of Physics/Univ of Crete) :

One is to use an alias in .bashrc (edit the file and add):

export FIG_DIR=/scisoft/share/star/bin/figaro
alias figaro='source $FIG_DIR/figaro.sh'

and another by adding the paths inside the Scisoft’s setup file (edit the file /scisoft/bin/Setup.bash as root). Change the following lines as shown:

#  Add the main Scisoft binaries to the command search path
if [ "X$PATH" != X ]; then
export PATH=/scisoft/bin:/scisoft/share/star/bin/figaro:$PATH
export PATH=/scisoft/bin:/scisoft/share/star/bin/figaro
#  Add the main Scisoft libraries to the library search path
if [ "X$LD_LIBRARY_PATH" != X ]; then
export LD_LIBRARY_PATH=/scisoft/lib:/scisoft/share/star/lib:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/scisoft/lib:/scisoft/share/star/lib

where the path to the FIGARO programs is added (using the “:”). Even without the libraries path (/scisoft/share/star/lib) FIGARO will start, but it seems better to include them as … you never know when a library is needed. To start FIGARO just type “figaro.sh” (.sh since we are using bash), after starting Scisoft (by using “. /scisoft/bin/Setup.bash” or any alias made already in .bashrc).

The difference in these two ways is that in the first case typing “figaro” will initiate the FIGARO program (and it will print the initialization message) while in the second case the initialization of Scisoft initializes FIGARO commands also (it will not print any messages but the commands are available).

The same can apply (most probably!) to any other program of STARLINK.

Scisoft 7.5 in Fedora 14 using bash

Scisoft is a great collection of astronomical software (IRAF 2.14-1, IDL 7.1, Python 2.5.4, PyRAF 1.9.0, and more) mainly used at the ESO. Installing the whole package is very easy through yum in Fedora, which is officially supported. Following the instructions you can have Scisoft ready in a few minutes and then what is left is to run the software.

For bash users type (“dot space” at the beginning):

. /scisoft/bin/Setup.bash

and csh/tcsh users:

source /scisoft/bin/Setup.csh

Most probably though the Scisoft will not start at all, as a check message about the SELinux is printed. Fedora comes with SELinux enabled by default which may create problems with some applications (like inmidas and PyRAF). The best solution is to disable this feature and to do so you have to edit the /etc/selinux/config (as root). What you will see is this text:

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
# SELINUXTYPE= can take one of these two values:
# targeted - Only targeted network daemons are protected.
# strict - Full SELinux protection.

where you change SELINUX= value from enforcing to disabled. Then if you run again the Scisoft will normally start and you can select you application by typing the appropriate keyword (like cl for IRAF, pyraf for PyRAF, etc).

There is still a warning message appearing about nm:
bash: nm: command not found...
Though it seems not to affect, it is better to go on and install the ‘binutils’ package (binutils collection of binary tools) and be sure that everything will be totally safe (go on and install it with yum install).

In order to avoid typing all the time this long command to start Scisoft an alias can be created easily at the .bashrc file, by adding this line:

alias scisoft='. /scisoft/bin/Setup.bash'

Then, just typing ‘scisoft’ is enough to invoke the programs.

Floating point numbers in bash or csh

They just don’t pass through!!

Bash or csh are unable to use floating numbers, as they understand numbers containing decimal point as strings. So, only actions with full numbers can be performed in these shells.