An HDF file is like a dictionary and it can store multiple objects. The to_hdf() function converts a Pandas object into an HDF file: 

df.to_hdf('store.h5', append = True, format='table')

When all the columns of a row are NaNs, they are not automatically dropped. This can be done by setting the dropna argument to True when writing to HDF.