i have date and month based folder are there thousand & lakhs of file there inside the folder s
what is good approach for that insert in to database in daily based like scheduler or service
i have date and month based folder are there thousand & lakhs of file there inside the folder s
what is good approach for that insert in to database in daily based like scheduler or service
Know the answer? Post it — somebody with the same question will find it here.
Sign in to answer this question
It is the same account you read, post and publish with — and you will come straight back to this page.
Jayraj ChhayaPosted Sep 26, 2025, 11:50 AM
Use a Python scheduler/service that runs daily, reads new files from the date/month folders, and does a bulk insert into MySQL (using
executemanyorLOAD DATA INFILEfor speed). After insertion, move files to an archive to avoid duplicates.