question for experts

Abd

Hello,
I have developped a tool using c#.net (windows application) that searches all files under subdirectories and get the files attributes (creation date, author, size, type...), and then it displays the data in a richtexteditor object...
my tool works fine: it takes few seconds for hundreds of files. but when testing thousands of files it takes hours, like for 10 000 files it takes arround 12 hours, for 30 000 files it takes more than 40 hours

is there a way you can advice to enhance my code structure? do i need to use threads? or anything like.. i know the application is consuming lot of memory while running

thanks for any advice

5 Replies

Know the answer? Post it — somebody with the same question will find it here.

Mahesh Chand15y ago
Avoid recursive if you can.
Put a try..catch..finally and Dispose oDetailedFileInfo inside the finally block.
0
Abd15y ago
Hello again,
I was rechecking my code, and i assume i found where the issue is.
In my code i am using
CFileInfo oDetailedFileInfo = new CFileInfo(fi);
which is creating an instance of this class each time the function is being called, and this is a recursive function... so for sure it is consuming lot of memory

so i guess the question is how to manage the memory here?
0
Ibrahim Ersoy15y ago
Its not surprise it would take too long.But if you use Parallel Programming it would take much less time.

You must absolutely use Parallel Programming case here.The links below will help you understand it.

http://aditiblogs.com/blog/blog/2009/06/07/parallel-programming-in-c-40-using-visual-studio-2010/
http://www.albahari.com/threading/part5.aspx
http://blogs.msdn.com/b/pfxteam/archive/2009/06/30/9809774.aspx

Good Luck!
0
Abd15y ago
thanks for your reply Felipe
i attached the helper class file
and the following is the function am using in the main:
        private void ProcessDir(string sourceDir)
        {
                // Process the list of files found in the directory.
                string[] fileEntries = Directory.GetFiles(sourceDir);
                foreach (string fi in fileEntries)
                {
                    try
                    {
                        // Read File Details from CFileInfo Object
                        CFileInfo oDetailedFileInfo = new CFileInfo(fi);
                        rtbResult.Text += oDetailedFileInfo.FileName + ";" + oDetailedFileInfo.FileAuthor + ";" + oDetailedFileInfo.FilePath + ";" + oDetailedFileInfo.FileCreationDate + ";" + oDetailedFileInfo.FileCreationDate2 + Environment.NewLine;
                    }
                    catch (Exception ux)
                    {
                        Log(Application.StartupPath + "\\Log.txt", DateTime.Now + Environment.NewLine + "Exception Error: " + ux.Message + Environment.NewLine + Environment.NewLine);
                        continue;
                    }
                }
                // Recurse into subdirectories of this directory.
                string[] subdirEntries = Directory.GetDirectories(sourceDir);
                foreach (string subdir in subdirEntries)
                    if ((File.GetAttributes(subdir) & FileAttributes.ReparsePoint) !=FileAttributes.ReparsePoint)
                        ProcessDir(subdir);
        }
0
Felipe Ramos15y ago
Abd it sounds like there is an issue in how you manage your files and the information, 10,000 is not a high enough count to slow down your application. I've an application to manage pictures and create thumbnails and so far I have over 50,000 images and I haven't seen an slow down yet. I would say to share a snippet of your most intensive code to see what we can do for you. If you don't feel comfortable look into the different types of algorithms and their linear interpretation to get a better idea of what would work better for you (anything square or cube you want to change).
0