[ad_1]
I have a pandas dataframe (created by appending several CSV files) with more than 5 million records. Since I need to use the data for a machine learning model, I want to convert it to JSON format so every time I open my ML code, the data loads faster. The code below is working fine without any error. However, it is taking a long time to execute. Basically, it is taking the same time as it would to read a huge CSV file. I believe one can read a JSON file with millions of records in a few seconds or minutes. Can anyone suggest how I can create and read the JSON file faster?
result = dfcustdata.to_json('custdata.json', indent= 1, orient="records")
dffinalcustdata = pd.read_json('custdata.json')
[ad_2]