Python: Reading dictionary from .txt, string keys and array type values

Question

I have saved my dictionary to a txt file as

f = open("dict_imageData.txt","a")
f.write( str(dict_imageData) + "\n" )
f.close()

And the saved txt file contains

{'002_S_0559': array([ 0.,  0.,  0., ...,  0.,  0.,  0.],dtype=float32)}
{'002_S_1070': array([ 0.,  0.,  0., ...,  0.,  0.,  0.], dtype=float32)}
{'023_S_0604': array([ 0.,  0.,  0., ...,  0.,  0.,  0.], dtype=float32)}

I have problem loading this dictionary and splitting keys and values. I tried splitting and also eval but did not work, since there is a dtype statement at the end.

Any suggestions?

Yes, I suggest you **not** to simply dumping the string representation of a dictionary to a text file and home-cooking your own serialization format, and instead, rely on one of the many standards that exist, i.e. JSON, YAML for text-based, `pickle` for binary. Since you are working with `numpy`, probably the best would be the `numpy` specific format available to you with `numpy.save` or [`numpy.savez`](https://docs.scipy.org/doc/numpy/reference/generated/numpy.savez.html#numpy.savez) for multiple named arrays... — juanpa.arrivillaga, Feb 06 '18 at 18:12
See also: https://stackoverflow.com/questions/36965507/writing-a-dictionary-to-a-text-file-in-python — pault, Feb 06 '18 at 18:18
Note: `eval` *would* work if the appropriate names are in scope, i.e. `array` and `dtype` and `float32`. But you definitely should consider one of the options mentioned above. — juanpa.arrivillaga, Feb 06 '18 at 18:25

score 1 · Accepted Answer · answered Feb 06 '18 at 21:38

Thanks for the comments, yes I am aware of using JSON and pickle, but I was more interested having a .txt file. I worked around my problem and found the solution as below:

First I define a MyEncoder class as:

class MyEncoder(json.JSONEncoder):
def default(self, obj):
    if isinstance(obj, np.integer):
        return int(obj)
    elif isinstance(obj, np.floating):
        return float(obj)
    elif isinstance(obj, np.ndarray):
        return obj.tolist()
    else:
        return super(MyEncoder, self).default(obj)

As my keys are strings and my values are arrays, before writing the key-value pairs I dump the values as

dict_imageData [key] = json.dumps(np.float32(np.array(values)), cls = MyEncoder)

Now, I write the dictionary to the file as

with open('dict_imageData.txt', 'a') as file:
    file.write(json.dumps(dict_imageData))
    file.write("\n")

For reading back the dictionary from the .txt file one I use eval

with open('dict_imageData.txt','r') as inf:
    lineno = 0
    for line in inf:  
        lineno = lineno + 1 
        print("line number in the file: ", lineno)
        dicts_from_file = dict()
        dicts_from_file = (eval(line))
        for key,value in dicts_from_file.items():
            print("key: ", key, "value: ", value)

Python: Reading dictionary from .txt, string keys and array type values

1 Answers1