I'm currently trying to write a dag to copy files from one S3 location to another. Before I can copy the files, I need to check the corresponding reject file to see if it is empty. The logic I need to follow is:
if rej_file_size = 0kb:
Copy files from S3_loc_1 to S3_loc_2
else:
Fail dag
I'm pretty new to dag and python, so what would be the best way for me to take a file from a location and see if it is empty (0kb) or not (>0kb)
Thank you for your time
Aucun commentaire:
Enregistrer un commentaire