Python3: read Julia's JLD format

Question

I've created some JLD files from Julia's dataframe module and now I'd like to import them into a Pandas dataframe. It's very easy to read and write these files in Julia, but I've not found the easy way in Python3. I've inspected the contents of the jld file and it is very complicated so there must be a package that will read into a dataframe, preserving column types, and names, etc. There is a JLD package for Python, but it appears to be for Python2 and has not been updated in 8 years, so I'm wonder what the current state of the art is in Python for reading JLD files.

Here is what I found:

https://pypi.python.org/pypi/jld/0.0.39

Why not save it as an HDF5? JLD is like HDF5 but I don't know if others can read it. — Chris Rackauckas, Jul 18 '17 at 05:33

kmsquire · Accepted Answer · 2017-08-01T06:31:33.377

3

How about writing the data frame out using Feather.jl:

import Feather

Feather.write("my_data_frame.feather", my_df)

and reading it into pandas with

import pandas

my_df = pandas.read_feather("my_data_frame.feather")

edited Aug 01 '17 at 06:31

answered Jul 19 '17 at 23:07

kmsquire

1,302
12
12

Thanks, will give it a try today. – Chuck Carlson Jul 20 '17 at 13:37
So far so good. I love these one line solutions. I thought I would get that with JLD but for some reason, the python JLD support for python3 never materialized. – Chuck Carlson Jul 20 '17 at 18:11

score 2 · Answer 2 · answered Jul 18 '17 at 09:54

julia's .jld is a "dialect" of hdf5 that is really designed for being read and written by julia itself -as it allows you to store type-related information. Just use plain HDF5 (there's a julia package for that) if you want to use anything across languages. -https://github.com/JuliaIO/HDF5.jl

NB: There will probably still be some work required to save your julia dataframe into a format that can be loaded easily into pandas. See here Pandas can't read hdf5 file created with h5py for some issues.

Python3: read Julia's JLD format

2 Answers2