1

right now I am able to open multiple netcdf files from a single folder using the command given below:

dsmerged = xarray.open_mfdataset('F:/netcdf/example/*.nc')

However, I am unable to open multiple netcdf files from different folders or directories using this command. Suppose I am having multiple netcdf files stored in multiple folders so how can I open together? Suggestions are appreciated.

Vishal Singh
  • 45
  • 3
  • 13

1 Answers1

1

From the docs, you can either pass in a glob string (like you have) or a list of explicit filenames to open. Therefore I would do the following:

import glob

# Get a list of all .nc files available in different folders
filenames = glob.glob("/parent/directory/*/*/*.nc")

dsmerged = xarray.open_mfdataset(filenames)

This works on Python 2.7 and 3.6.

Note you may have to run this a few times and concatenate the returned lists if not all files are in the same directory structure. I.e. if some .nc files are in /path/one/here/file.nc and others are in /path/here/file.nc

tda
  • 2,045
  • 1
  • 17
  • 42
  • Above answer looks good. May have to merge append and concat if you are trying to merge along more than one dimension as I was trying to do https://stackoverflow.com/questions/47545138/python-xarray-open-mfdataset-concat-along-two-dimensions – Ray Bell May 01 '18 at 15:00
  • If you do two glob.glob() calls to two different directories, which returns two lists with .nc files, a simple finallist = list1+list2 should concatenate to form a complete list of relevant files. – tda May 01 '18 at 15:06
  • That's not a very useful comment if you want further help haha. This works fine for me on Python 2.7 and 3.6 so could you please elaborate on what issues you're having please? What error is being returned? – tda May 02 '18 at 07:46
  • i am using winpy 3.6 only and when i running it i am getting error " File "C:\python3\WinPython\python-3.6.5.amd64\lib\site-packages\xarray\core\dtypes.py", line 186, in result_type return np.result_type(*arrays_and_dtypes) TypeError: invalid type promotion – Vishal Singh May 02 '18 at 08:00
  • TD-Asker @ even i tried this too " ds = xarray.open_mfdataset("/parent/directory/*/*/*.nc") " but getting the same error however, if i m trying this command with only one folder so it is working – Vishal Singh May 02 '18 at 08:03
  • Have you tried running the command with each folder individually? There may be an issue with one .nc file in a specific directory, and this is causing the error? – tda May 02 '18 at 08:07
  • TD-Asker@ Yes, you were right, now it's working, there may b a problem in any netcdf file. Many thanks for your help. – Vishal Singh May 02 '18 at 08:15