about Grigoris Maravelias
Buggy behaviour of unpack within numpy.loadtxt

Buggy behaviour of unpack within numpy.loadtxt

The routine loadtxt from numpy (numpy.loadtxt) can be used to load columns of data from various files. This works great as long as numbers exist and becomes buggy with strings.

Suppose we have a file (named ‘test.test’) with these values:

3103725.  1616.93596535  13.656  0  2011-05-23T23:49:35  
3139474.  1405.95436047  13.643  0  2011-05-23T23:51:16  
3026925.  1370.07921223  13.683  0  2011-05-23T23:54:40 
...

First, if we want to assign many values we have to use unpack parameter, like:
>>> x, y, z = loadtxt('test.test', unpack=True, usecols=(0,1,2))

and if one of these columns is strings then we should define this by using dtype:
>>> x, y, z = loadtxt('test.test', unpack=True, dtype=(float,float,'S19'), usecols=(0,1,4)) or
>>> x, y, z = loadtxt('test.test', unpack=True, dtype={'names':['n1','n2','s1'],'formats':['f','f','S19']}, usecols=(0,1,4))

and of course we should get 3 columns with the the third one to be a column of strings.

But … this will not happen! The unpack parameter is buggy up to 1.3.0 version (at least) and it has been corrected at a later version (it works after version 1.6 for sure). So since I didn’t have that version (and I didn’t want to spend time upgrading) a solution was given by Derek H. within AstroPy list. The unpack parameter is not used at all and the assignment is done by this workaround:
>>> a = loadtxt('test.test',dtype=[('x','f'),('y','f'),('z','S19')],usecols=[0,1,4])
x,y,z = a['x'], a['y'], a['z']

Leave a Reply

Your email address will not be published. Required fields are marked *