![]() ![]() # Function converts the json string to a dataframe and appends it directly to the CSV fileĭef add_json_string_to_csv(api_result_string):ĭf_with_new_data.to_csv('output.I have a scenario, I am receiving logs in csv format to kinesis firehose directly and with help of lambda function (python) I need to transform the csv logs format to json format and return the output back to kinesis. The advantage is that all data is already in the CSV if the program crashes or if you terminate it. Instead of building a huge dataframe over time, the code below appends the fetched data directly into the CSV file. The output of above code: item amount weight price bestbeforeendeateĮDIT: I had another look at the problem and thought I share another solution, which might be better for you. # The dataframe you want to store everythingĭf = add_new_entry_to_dataframe(df, newly_fetched_result) Input_parsed = json.loads(api_result_string)ĭf_with_new_data = pd.json_normalize(input_parsed) # Function converts the api result to the dataframe and appends it to dfĭef add_new_entry_to_dataframe(df, api_result_string): In order to convert a JSON string into a dict you can simply use the json library. ![]() Pandas has a function called json_normalize, which can directly convert a dict into a dataframe.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |