Archive for November, 2011

matplotlib’s pie colors or why pie diagrams suck!

Posted November 30, 2011 By grigoris

While asking around for the best solution in how to determine the colors inside the matplotlib’s pie function I found out that the best solution is just … not to use them at all! (At least if they are not more than a couple of objects!)

Well, the problem starts from the fact that when pie assigns the colors automatically then black (one of the predefined colors) covers the text (black also) and the percentage of the specific part cannot be seen. So the question is how to remove this? One answer (thanks to Tony Y.) included to set the rcParams by typing:

>>> import matplotlib.pyplot as plt
>>> plt.rcParams['axes.color_cycle'].remove('k')

but for some reason**, although the black was removed from the list, the pie kept using the black! Moreover the color_cycle is limited to 8 colors ([‘b’, ‘g’, ‘r’, ‘c’, ‘m’, ‘y’, ‘k’]) so after the 8th part the colors start repeating again, so actually no point… Another solution is to give specifically the colors (like colors=(‘g’,’r’,’b’,’y’,’c’,’m’,’#e23f5f’,’#eeefff’, ‘#ffee88’) inside the pie) but this works if you know the number of pie parts, unless you create a more dynamic way to generate colors.

But this is probably getting too much (if not useless and boring…) and moreover if there are many objects then the pie diagram does not help to understand what it represents. Che M. pointed out a very interesting page which explains really easily why pie diagrams should just be … forgotten totally and replaced by bar plots which are by far a better way to present (and comment easily) data.

So finally I decided to follow this path… ignore pie and go on with the bar plots!

>>> added at 6 Dec 2011 >>>
** It is not removed because it is actually hard-coded in the function (pointed out by Tony Y.).

Another approach would be to change the way that the text appears and this is successful by using this code (by Jae-Joon L.):
a = pie([1,2,4,3], labels=["1", "2", "4", "3"], autopct="%d")
from matplotlib.patheffects import withStroke
effect_whiteborder = withStroke(foreground="w", linewidth=3)
for txt in a[2]:
    txt.set_path_effects([effect_whiteborder])

which works and the text is obvious over black color.

IRAF’s error message 502 and pixel value replacement

Posted November 25, 2011 By grigoris

While running a data reduction of IR observations with IRAF, the ccdproc printed this error:

ERROR (502, “floating point invalid operation”)

which means that some values of the image used (the flat image in this case) are not suitable for the performing mathematical actions (like division with 0s). Moreover the IR ccds have many bad pixels so the best is to replace these values with standard ones in order to know exactly which areas are problematic.

In order to replace them the imcalc command (stsdas / toolbox / imgtools) can be used to replace these values, like:
imcalc master_flat_temp.fits output="master_flat.fits" equals="if (im1.le.0.0) then -9.0 else im1"
if we want for example to replace all values lower or equal (le) to 0.0 with -9.0.

Flatcombine and Darkcombine (based on imcombine both) in IRAF just combine the images given as input to create a master flat / dark image without performing any normalization. This is done when using ccdproc with the parameters flatcor and darkcor set to yes.

IRAF’s error message 1121 with hedit

Posted November 24, 2011 By grigoris

Although hedit of IRAF is supposed to handle the keywords of a fits file (add, remove) its behaviour can be misleading sometimes. An error like this:

ERROR (1121, “FXF: EOF encountered while reading FITS file (something.fits)”)

means that after altering the keywords in the header the EOF of the header has been placed wrongly. The solution is to place some dummy keywords more… but it seems to be an insisting bug.

Buggy behaviour of unpack within numpy.loadtxt

Posted November 8, 2011 By grigoris

The routine loadtxt from numpy (numpy.loadtxt) can be used to load columns of data from various files. This works great as long as numbers exist and becomes buggy with strings.

Suppose we have a file (named ‘test.test’) with these values:

3103725.  1616.93596535  13.656  0  2011-05-23T23:49:35  
3139474.  1405.95436047  13.643  0  2011-05-23T23:51:16  
3026925.  1370.07921223  13.683  0  2011-05-23T23:54:40 
...

First, if we want to assign many values we have to use unpack parameter, like:
>>> x, y, z = loadtxt('test.test', unpack=True, usecols=(0,1,2))

and if one of these columns is strings then we should define this by using dtype:
>>> x, y, z = loadtxt('test.test', unpack=True, dtype=(float,float,'S19'), usecols=(0,1,4)) or
>>> x, y, z = loadtxt('test.test', unpack=True, dtype={'names':['n1','n2','s1'],'formats':['f','f','S19']}, usecols=(0,1,4))

and of course we should get 3 columns with the the third one to be a column of strings.

But … this will not happen! The unpack parameter is buggy up to 1.3.0 version (at least) and it has been corrected at a later version (it works after version 1.6 for sure). So since I didn’t have that version (and I didn’t want to spend time upgrading) a solution was given by Derek H. within AstroPy list. The unpack parameter is not used at all and the assignment is done by this workaround:
>>> a = loadtxt('test.test',dtype=[('x','f'),('y','f'),('z','S19')],usecols=[0,1,4])
x,y,z = a['x'], a['y'], a['z']

Replacing many characters in a string with Python

Posted November 4, 2011 By grigoris

In order to replace a character in a string with Python replace command can be used easily: str.replace(old, new[, count])
[return a copy of the string with all occurrences of substring old replaced by new. If the optional argument count is given, only the first count occurrences are replaced.]

But if we want to change more than one character then we have to do it differently. A nice solution I found at Gomputor’s blog is by defining a function like this:

def replace_all(text, dic):
	for i, j in dic.iteritems():
		text = text.replace(i, j)
	return text

So then we just have to give our string (text) and the list of changes (dic) like:st = '2011-05-23T22:31:35'
chgs = {'-':'','T':'.',':':''}

and use that function: >>>print st, replace_all(st,chgs)
2011-05-23T22:31:35 20110523.223135