Tag Archive


amateur astronomy awk bash be b[e] supergiant cartoon conference convert evolved star exoplanet fedora figaro fits fun galaxy history iraf jupiter latex linux lmc machine learning massive star matplotlib meteor mypaper paper peblo photometry planet pro-am pyraf python scisoft skinakas observatory small magellanic cloud smc spectroscopy starlink talk theli ubuntu university of crete video x-ray

Merging catalogs and creating unique identifier in bash

For a certain project I had created a number of photometric catalogs, each one corresponding to a specific observing field. I would like to construct the final (merged) one but for this I needed to add a unique source identifier at the beginning of each row. I decided to create a F#-**** tag for each source with “F#” corresponding to the field id and **** to a counter for each source per field. The final command was:

for i in {1,2,4,5,6,7,8,9,10,11,12,13,16};do echo F$i.matches.all.cat;awk -v id="$i" 'FNR>1 {print "F"id"-"1+c++, $0}' F$i.matches.all.cat >> results.tmp; done

So the command reads all the specific numbers for which a catalog with a filename of F*.matches.all.cat exists. The number of each field ($i) is parsed as an external variable (id) to awk which places it as the unique identifier “Fid-counter” with the incremental “counter” (1+c++) corresponding actually to the number of row (1+counter to begin from 1 instead of 0 – FNR avoids the first line of each catalog which is a column description). All results are written appended to the output file results.tmp (created automatically when non-existing).

Then, we can use sed to add the header:

sed -i '1i\#SourceID ...' results.tmp