2

the following sourcecode produces a memoryerror on my machine:

import numpy as np
x = np.random.random([100,100,100])
y = np.random.random([100,100,100])
c_sort = np.argsort(x, axis = 2)
f = y[c_sort]

Do you have a nice and easy idea how to avoid the memory error?

The other way to do this is

x = np.random.random([100,100,100])
y = np.random.random([100,100,100])
f = np.zeros([100,100,100])
for i in range(100):
    for j in range(100):
        f[i,j,:] = y[i,j, np.argsort(x[i,j,:])]

But I wonder why the solutions above does not lead to the same result?

varantir
  • 6,624
  • 6
  • 36
  • 57
  • What are you trying to do? – Ignacio Vergara Kausel Jan 09 '18 at 15:01
  • Hi Ignacio, I'm trying to sort y accoding to the order of (the second axis) of x. – varantir Jan 09 '18 at 15:06
  • Are you sure you want `y[c_sort]` for that? Do it on a small example and see if that's what you really need. You might be looking to do advanced-indexing, as shown here - https://stackoverflow.com/questions/46103044/index-n-dimensional-array-with-n-1-d-array – Divakar Jan 09 '18 at 15:13
  • Dear Divakar, I am totally sure I need y[c_sort]. I want the *whole* ndarray to be sorted according to the second axis. – varantir Jan 09 '18 at 15:15
  • Well then you would have an output of shape `(100, 100, 100, 100, 100)`. Can your system RAM handle that? – Divakar Jan 09 '18 at 15:19
  • @Divakar: Okay, maybe I am mistaken by the syntax... – varantir Jan 09 '18 at 15:21
  • 2
    So, which one's the correct version? The `y[c_sort]` one would produce different result I think. If the loopy one is the correct one, then the earlier linked post should solve your case, if you were looking to optimize. – Divakar Jan 09 '18 at 15:26
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/162827/discussion-between-varantir-and-divakar). – varantir Jan 09 '18 at 15:30

1 Answers1

1

After discussion in the comments, it seems the loopy version is the correct one. So, to optimize it, we can use advanced-indexing. Thus, given the argsort indices as idx = np.argsort(x,axis=2), we can have f like so -

m,n = y.shape[:2]
f = y[np.arange(m)[:,None,None], np.arange(n)[:,None], idx]

Generic helper function for advanced-indexing take_along_axis could be useful.

Divakar
  • 218,885
  • 19
  • 262
  • 358