Introduction

I've come across a requirement, on a number of occasions, to produce a Windows application that interacts with a remote web server. The Windows application may be dealing with a web-service, or simply automating form input or screen scraping - what is common, is that there is one side of the equation that is web based and therefore, can potentially handle multiple requests, allowing us to complete the process faster. This article is a project that consists of two parts: a threaded Windows client, and a simple MVC application that it interacts with. As not all remote services allow multiple connections, the project gives the option for running the process in sequential or parallel (threaded) mode. The source code of both projects is attached.

Background

The main concept being introduced in this article is Windows Multi-threading. The approach I have taken is one of many open to developers.

The technologies being demonstrated here are.

The threading concept is as follows.

Setting things up

The simple server - An MVC app

In order to test our work, and not trigger a denial of service warning (large number of multiple threads can do that!), we will create a test harness. In this case, a simple MVC application will suffice. We will create a controller method GetXML that takes in an ID sent by the Windows application, and returns an XML response.

The GetXML method is called like this: http://localhost:4174/home/GetXML?ItemID=23.

And returns output XML like this.

  1. <response type="response-out" timestamp="20130804132059">
  2. <itemid>23</itemid>
  3. <result>0</result>
  4. </response>

NB: for the purposes of this test, a "result" of 0 = failure, 1 = success.

Create a new MVC app, and add a new controller GetXML. We are also going to put a small "sleep" command to slow things down a bit and emulate delay over the very busy Interwebs.

  1. public ContentResult GetXML()
  2. {
  3. // assign post parameters to variables
  4. string ReceivedID = Request.Params["ItemID"];
  5. // generate a random sleep time in milli-seconds
  6. Random rnd = new Random();
  7. // multiplier ensures we have good breaks between sleeps
  8. // note that with Random, the upper bound is exclusive so this really means 1..5
  9. int SleepTime = rnd.Next(1, 2) * 1000;
  10. // generate XML string to send back
  11. System.Threading.Thread.Sleep(SleepTime);
  12. return Content(TestModel.GetXMLResponse(ReceivedID), "text/xml");
  13. }
Create a model method that takes care of the logic of constructing the XML response to send back. This will take as parameter an ID (int) that represents the identifier of a list of objects/queries that the user is working with. These could be credit cards, websites to scrape, account numbers, etc. In reality, you can send any data you need to work with, and return, then, to the main calling application.
  1. public static string GetXMLResponse(string ItemID)
  2. {
  3. // generate a random result code, 0= fail, 1 = success
  4. // note that with Random, the upper bound is exclusive so this really means 1..2
  5. Random rnd = new Random();
  6. string ResultCode = rnd.Next(0, 2).ToString();
  7. string TimeStamp = GetTimeStamp();
  9. // create an XML document to send as response
  10. XmlDocument doc = new XmlDocument();
  12. // add root node and some attributes
  13. XmlNode rootNode = doc.CreateElement("response");
  14. XmlAttribute attr = doc.CreateAttribute("type");
  15. attr.Value = "response-out";
  16. rootNode.Attributes.Append(attr);
  17. attr = doc.CreateAttribute("timestamp");
  18. attr.Value = TimeStamp;
  19. rootNode.Attributes.Append(attr);
  20. doc.AppendChild(rootNode);
  22. // add child to root node sending back the item ID
  23. XmlNode dataNode = doc.CreateElement("itemid");
  24. dataNode.InnerText = ItemID;
  25. rootNode.AppendChild(dataNode);
  27. // add our random result
  28. dataNode = doc.CreateElement("result");
  29. dataNode.InnerText = ResultCode;
  30. rootNode.AppendChild(dataNode);
  32. // send back xml
  33. return doc.OuterXml;
  34. }

The threaded client - A Windows form app

The client is visually quite simple. It contains two edit boxes for input variables, a "listview" to show the user what is happening, and a "checkbox" to tell the program if it should run in sequential or threaded mode.

We will go through the overall logic first by examining the sequential process, and then look at the threading part. At the top of the form class, we keep track of some variables.

  1. private int RunningThreadCount;
  2. private int RunTimes;
  3. private int TimeStart;
Everything is kicked off by the RunProcess button click event.
  1. TimeStart = System.Environment.TickCount;
  2. InitProcess();
  3. if (chkRunThreaded.Checked)
  4. RunProcessThreaded();
  5. else RunProcess();
We keep track of the start time, and update this when all processes are complete to test how long the process took. We also, at this stage, call an Init method that sets things up for us, assigning some variables and filling the ListView with values.
  1. // set up some default values
  2. public void InitProcess()
  3. {
  4. btnExit.Enabled = false;
  5. btnRunProcess.Enabled = false;
  6. chkRunThreaded.Enabled = false;
  7. RunTimes = int.Parse(edtTimesToRun.Text);
  8. FillListView();
  10. RunningThreadCount = 0;
  11. }
  13. // fill the ListView with the count items
  14. public void FillListView()
  15. {
  16. lvMain.Items.Clear();
  17. for (int i = 0; i < RunTimes; i++)
  18. {
  19. ListViewItem itm = new ListViewItem();
  20. itm.Text = (i+1).ToString();
  21. itm.SubItems.Add("Pending");
  22. itm.SubItems.Add("-");
  23. itm.SubItems.Add("-");
  24. lvMain.Items.Add(itm);
  25. }
  26. }
Let's now look at the sequential RunProcess method. This controls the main body of work for each web request. The number of times to run the process is set by the value of edtTimesToRun.Text which is assigned to the variable RunTimes.

The RunProcess method has a keyword of async - this is important as we are using the await keyword within the RunProcess method. The important part of this code is SendWebRequest - this takes the input, queries the web server, and returns a value that we use to update the UI for the user.
  1. public async void RunProcess()
  2. {
  3. for (int i = 0; i < RunTimes; i++) {
  4. updateStatusLabel("Processing: " + (i + 1).ToString() +
  5. "/" + RunTimes.ToString());
  6. lvMain.Items[i].Selected = true;
  7. lvMain.Items[i].EnsureVisible();
  8. lvMain.Items[i].SubItems[1].Text = "Processing...";
  9. SimpleObj result = await Shared.SendWebRequest(
  10. new SimpleObj()
  11. { ItemID = i.ToString(),
  12. WebURL = edtTestServer.Text}
  13. );
  14. lvMain.Items[i].SubItems[1].Text = result.ResultCode;
  15. if (result.ResultCode == "ERR")
  16. lvMain.Items[i].SubItems[2].Text = result.Message;
  17. }
  18. CleanUp();
  19. }
SendWebRequest is located in a separate shared.cs file as it is used from two different places. The shared.cs file also contains a simple object called SimpleObj. This is used to carry data between methods.
  1. public class SimpleObj
  2. {
  3. public string WebURL; // web address to send post to
  4. public string ResultCode; // 0 = failure, 1 = success
  5. // Used to store the html received back from our HTTPClient request
  6. public string XMLData;
  7. public string Message; // What we will show back to the user as response
  8. public string ItemID;
  9. }
  10. // Used to store the ListView item ID/index so
  11. // we can update it when the thread completes
The SendWebRequest method is again, flagged as async. This will be explained later.

In the SendWebRequest method, we set up an HTTPClient, calling its PostAsync method. Here, we are telling the HTTPClient to perform a "POST" action against the Server. If you have done web programming before, you will recall setting up a form,
  1. <form action="somedomain.com/someaction?somevalue=134" method="post">
  2. <input type="text" id="ItemID">
  3. <input type="submit" value="send">
  4. </form>
That is what, in effect, we are doing here. We create the client object, send in as parameters the URL of the website we want to send the data to, together with the "content", which is the data packet to send. In this case, the content simply consists of the parameter ItemID and its value.
  1. HttpResponseMessage response = await httpClient.PostAsync(rec.WebURL, content);
The await keyword tells the code to sit there until with the HTTPClient comes back with a response, or an exception is raised. We examine the response to ensure it is valid (response.IsSuccessStatusCode), and assuming it is, we proceed to take the response content stream and process its XML result. Note the outer Try/Except wrapper - this will catch any HTTP connection errors and report these separately and allow the application to continue working smoothly.
  1. public static async Task<SimpleObj> SendWebRequest(SimpleObj rec)
  2. {
  3. SimpleObj rslt = new SimpleObj();
  4. rslt = rec;
  5. var httpClient = new HttpClient();
  7. // we send the server the ItemID
  8. StringContent content = new StringContent(rec.ItemID);
  9. try
  10. {
  11. HttpResponseMessage response =
  12. await httpClient.PostAsync(rec.WebURL, content);
  13. if (response.IsSuccessStatusCode)
  14. {
  15. HttpContent stream = response.Content;
  16. Task<string> data = stream.ReadAsStringAsync();
  17. rslt.XMLData = data.Result.ToString();
  18. XmlDocument doc = new XmlDocument();
  19. doc.LoadXml(rslt.XMLData);
  20. XmlNode resultNode = doc.SelectSingleNode("response");
  21. string resultStatus = resultNode.InnerText;
  22. if (resultStatus == "1")
  23. rslt.ResultCode = "OK";
  24. else if (resultStatus == "0")
  25. rslt.ResultCode = "ERR";
  26. rslt.Message = doc.InnerXml;
  27. }
  28. }
  29. catch (Exception ex)
  30. {
  31. rslt.ResultCode = "ERR";
  32. rslt.Message = "Connection error: " + ex.Message;
  33. }
  34. return rslt;
  35. }

So, that is the basic sequential work flow. Take a list of work items, iterate through them in sequence, call the web server, and parse back the XML response.

As the processes are run sequentially, the overall time taken to complete can be high.

Now, let's run through the RunProcessThread code and see the difference.

Here is our outer wrapper method in the main form.
  1. public void RunProcessThreaded()
  2. {
  3. updateStatusLabel("Status: threaded mode - watch thread count and list status");
  4. lblThreadCount.Visible = true;
  6. for (int i = 0; i < RunTimes; i++)
  7. {
  8. updateStatusLabel("Processing: " + (i + 1).ToString() + "/" + RunTimes.ToString());
  9. lvMain.Items[i].Selected = true;
  10. lvMain.Items[i].SubItems[1].Text = "Processing...";
  11. SimpleObj rec = new SimpleObj() { ItemID = i.ToString(), WebURL = edtTestServer.Text };
  12. CreateWorkThread(rec);
  13. RunningThreadCount++;
  14. UpdateThreadCount();
  15. }
  16. }
The critical change is that instead of carrying out the WebRequest task on each loop sequentially, we are passing that task off to the method CreateWorkThread, passing in the required parameters.
  1. public void CreateWorkThread(SimpleObj rec){
  2. ThreadWorker item = new ThreadWorker(rec);
  3. //subscribe to be notified when result is ready
  4. item.Completed += WorkThread_Completed;
  5. item.DoWork();
  6. }
This small method creates a new object ThreadWorker, and tells it to call WorkThread_Completed when it is finished. The WorkThread_Completed method in the form simply updates the form UI *in the context of the form thread* and performs some cleanup.
  1. //handler method to run when work has completed
  2. private void WorkThread_Completed(object sender, WorkItemCompletedEventArgs e)
  3. {
  4. lvMain.Items[int.Parse(e.Result.ItemID)].SubItems[1].Text = e.Result.ResultCode;
  5. if (e.Result.ResultCode == "ERR")
  6. lvMain.Items[int.Parse(e.Result.ItemID)].SubItems[2].Text = e.Result.Message;
  8. RunningThreadCount--;
  9. UpdateThreadCount();
  11. if (RunningThreadCount == 0)
  12. {
  13. CleanUp();
  14. }
  15. }
I have created a separate class/file ThreadWorker to manage the thread work - this keeps things separate and clean.

The class has some private members and a public event.

  1. class ThreadWorker
  2. {
  3. private AsyncOperation op; // async operation representing the work item
  4. private SimpleObj ARec; // variable to store the request and response details
  5. public event EventHandler<WorkItemCompletedEventArgs> Completed;
  6. //event handler to be run when work has completed with a result
  9. // constructor for the thread. Takes param ID index of the Listview to keep track of.
  10. public ThreadWorker(SimpleObj Rec)
  11. {
  12. ARec = Rec;
  13. }
  14. }
You will recall that in our main form, when we are creating each thread worker, we set up the thread object, then tell it to DoWork so this is the main kick-off method for the thread object.
  1. public void DoWork()
  2. {
  3. //get new async op object calling forms sync context
  4. this.op = AsyncOperationManager.CreateOperation(null);
  5. //queue work so a thread from the thread pool can pick it up and execute it
  6. ThreadPool.QueueUserWorkItem((o) => this.PerformWork(ARec));
  7. }
The reason I am using a ThreadPool object is that creating threads is a very expensive operation. Therefore, using the pool means that on completion, threads can be put into a pool to be reused. After being added to the pool, we tell the thread to kick off the method PreformWork. This method calls our shared method SendWebRequest, and when that is finished, calls PostCompleted which gets picked up by the WorkThread_Completed method in the main form class.
  1. private void PostCompleted() //SimpleObj result
  2. {
  3. // call OnCompleted, passing in the SimpleObj result to it,
  4. // the lambda passed into this method is invoked in the context of the form UI
  5. op.PostOperationCompleted((o) =>
  6. this.OnCompleted(new WorkItemCompletedEventArgs(ARec)), ARec);
  7. }
  9. protected virtual void OnCompleted(WorkItemCompletedEventArgs Args)
  10. {
  11. //raise the Completed event in the context of the form
  12. EventHandler<WorkItemCompletedEventArgs> temp = Completed;
  13. if (temp != null)
  14. {
  15. temp.Invoke(this, Args);
  16. }
  17. }
Our main form "WorkThread_Completed" method watches each incoming terminated thread, and when they have all completed, runs some cleanup code.
  1. //handler method to run when work has completed
  2. private void WorkThread_Completed(object sender, WorkItemCompletedEventArgs e)
  3. {
  4. lvMain.Items[int.Parse(e.Result.ItemID)].SubItems[1].Text = e.Result.ResultCode;
  5. if (e.Result.ResultCode == "ERR")
  6. lvMain.Items[int.Parse(e.Result.ItemID)].SubItems[2].Text = e.Result.Message;
  7. RunningThreadCount--;
  8. UpdateThreadCount();
  9. if (RunningThreadCount == 0)
  10. {
  11. CleanUp();
  12. }
  13. }

And that is it. As you can see, running threads adds a bit more code, but dramatically improves performance.

As I stated at the start of this article, this is but one way of handling a threaded application. Thread pools have their advantages and disadvantages, you need to weigh up your goals and granular needs against the ease of use. If you are interested in this area, you should also look at Background worker and if you want to harness the power that is in multi-core CPUs while threading the Task Parallel Library is a great way to go.

(PS - If you found this article useful or downloaded the code, please let me know by giving a rating below!)

Happy threading!