python - Finding n nearest data points to grid locations -

I am working on a problem where I have a large group (& gt; 4 million) - digital space,

With scalar function value in each. It is represented by four arrays: XD, YD, JDD, and FD. Tupal (xd [ii], wd [ii], jdd [i]) refers to the location of the data point ii, in which the value of the FD [ii] is.

I want to make irreversible 100x100x100 points in the same place as my data say, a resinine grid. This grid is set as follows. XGrid, YGrid, ZGrid] = np.mgrid [Xmin: Xmax: Xstep, Ymin: Ymax: Ystep, Zmin: Zmax: Zstep] XG = XGrid [,, 0,0] YG = YGrid [0::, 0] ZG = ZGrid [0,0,:]

XGrid is a 3D array of X-val, each point in the grid is a 1D array of xmx from xzi x-x that is xstep Separates from the distance.

I want to use a projection algorithm, I want to find the value of the function on each grid point based on the data around it. In this algorithm I need the closest (or at least approximate) data of 20 Gate-Positive Interests. It is, for the grid point (XG [i], YG [J], ZG], I have 20 nearest data I want to find points.

The only way I can think, goes through every data point for the loop, and later all (many!) Loops going through data points Is embedded for, Eucl Calculates Diane distance, and selects the nearest 20 person.

  in the category i (0, XG.shape): for the range in Jammu (0, YG.shape): Category (0, ZG.shape) for: distance = np.zeros ([XD.shape]) for a range (0, XD.shape): distance [A] = (xd [a] - xg [ii]) ** A limit for 2 + (YD [A] - YG [J]) ** 2 + (JDD [A] - ZG [K]) 2b = NP joros ([20], int) 020): Indx = N.P. Regeneration (Distance) B [A] = Indx Distance [Index] = Float (Inf)   
 This gives me the closest data to the grid point An array of index Induon will b. I think it will take a lot of time to go through each data point at each grid point.  
 I am looking for any suggestions, such as how can I be able to organize data points before calculating distance, which can be cut in time of calculation.   
 
 
 There is no need to iterate your data points for each grid space: the natural of your grid locations The order has been ordered, so just repeat your data points  once , and assign each data point to eight grid places that surround it when you do, in some grid space There may be very little data points. Check the data points of the adjacent grid locations if you have enough data points (how your data is distributed), you can already select the nearest 20 closest neighbors.  
  Appendix:  You can also reconsider other parts of your algorithm. Your algorithm is a type of piece-line-linear interpolation, and there are a lot more simple improvements, instead of splitting your position into equally distant cubes, consider assigning multiple center points and dynamically reducing them. Consider transferring to as long as the average distance of the data points from the nearest center point is minimal, in this way:  
 
 allocate every date Point will minimize your nearest center point indicates  
 Repeat for each focal point to coordinate, which is the average distance from "their" points (data subset "centroid").  
 Some data points are now a different closest center point, step 1 and repeat. Unless you are united (or near enough).