Besides understanding the more mundane aspects of C# and the .NET framework, you need to be well familiar with .NET REMOTING, Database (MS SQL), .NET Cryptography, SOAP, XML, ASP.NET and ADO.NET to follow this article and the corresponding code easily. Familiarity with the financial world is advantageous.
Objective
- The advent of C# and .NET has alleviated the design and development of applications to a completely new and MUCH BETTER!! Platform.
- In this article I will discuss some of the design and development issues that one might consider when using .NET framework for developing Network affiliated applications.
The sample application is developed with the following issues in mind:
- Develop reusable components including the REMOTING and Cryptography modules;
- Provide easily configurable framework for local network access and develop components that could easily be reused to develop web services and applications accessible over the internet; and
- Provide a framework that better addresses performance and post release maintenance and enhancement issues.
Technology Coverage
- The implementation of the sample has touched almost all aspects of C# and .NET. These mainly include:
Serialization, Marshalling By Value and By Reference (MBV and MBR), Call Contexts, Threading, Events, Delegates, Reflection, Custom Attributes, Unsafe code (for including C++ style memory access), Remoting, Cryptography, UI and Configuration.
- For obvious reasons of space and for the sake of avoiding repetition, the detailed account of the underlying technology behind the code is not discussed. The discussion focuses on the design and implementation issues surrounding the overall framework of the application.
Overall Layout
- The application contains three major components; namely; Client side components, Server side components (remote objects) and secured Transport and Messaging components.
- The overall relationship between these components is depicted below:
The components are organized in the following assemblies:
Client Side Components
- NefsWIndowsClient: This is a Multi Document Interface (MDI) assembly that includes the main user interface routines and helper objects, namely; Event Monitor, Event Manager (in NeFsModuleManager.cs file), Menu Builder and other utilities in NeFsWinUtilities.cs file.
- NeFsClientDataAccessComponent: This assembly includes objects that easily map the underlying data access routines to the user interface. (More about it latter in the Data Access design section) .
Remoting and Cryptography
- NeFsCryptography: The assembly includes the main RSA and Message Digest 5 (MD5) based crypto service provider. It also includes public/private key manger that allows the user to generate and manage public/ private keys used to digitally sign messages during encryption and enable to verify decryption processes for message-key integrity during channeling messages across the network.
- NeFsSerializableLogicalObjects: The assembly contains Serialisable and Call Context objects. These objects are included to enhance client-server communication.
- NeFsRemoting: The client/server channel sink and formatter providers and the corresponding channel and message formatter sinks are included in this assembly. In addition to this, the assembly also includes an interface that is used as a metadata by the client to identify the underlying accessible methods and fields of the remote object.
Server Side Components
- NeFsServer: This is an executable console program that initialises the server.
- NeFsAnalytics: This assembly contains the basic option pricing models included in the sample application. These are: Trinomial option pricing model for pricing American Options, Monte Carlo Simulation and Black Sholes pricing models for evaluating and pricing European type options.
- NeFsDataAccess: This assembly includes the underlying data access objects and methods.
- NeFsRemoteObject: This assembly includes the underlying remote object. The object is a MBR object. It is currently configured as a WellKnown object. It could easily be configured to be either Client Activated or Singleton.
Other Helper Classes
- NeFsTrace: Provides Console, Windows Event Log and File based trace outputs.
- NeFsUtility: Provides I/O manipulation, random generator and secret key comparer helper methods.
The User Interface (NefsSystem.WindowsClient)
- The user interface is organized into different controls mapping to the services provided by the server. The main object of the current exercise is to show how accessing remoting components could be accomplished using .NET framework. The GUI is designed with this object in mind.
- The interface includes two different levels of access menus. The first level enables the user to administer users, manage keys, manage configuration and allow to login to the system. The second level provides a direct access to the main application components of the remote server.
- The relationship amongst the different components of the user interface is managed using tailored classes NeFsEventArgs, NeFsEventMonitor, NeFsEventManger, BuildMenu, MenuArgs,and NeFsGraphicHelper. I will leave the details for you to go through it as most of the stuff is very straightforward.
The Remoting and Cryptography Assemblies
These assemblies are written as an independent component and can easily be plugged into other networked applications that make use of the secured Remoting architecture of the .NET framework.
- The remoting component includes the following client/server classes:
- The .NET framework remoting architecture largely influences the implementation of these classes.
- The sink and format providers implement IClientFormatterSinkProvider,IClientChannelSinkProvider and IServerFormatterSinkProvider, IServerChannelSinkProvider of .NET respectively.
- The corresponding client channel and formatter sinks implement IClientFormatterSink,IMessageSink,IClientChannelSink,IChannelSinkBase interfaces. While the corresponding server side sinks implement IServerChannelSink interface.
- The implementation of the classes is largely the same across both client and server. For the sake of illustration I will discuss the server side code in brief below:
- When the server is initialized via NeFsServer.exe application it picks the following runtime information from the configuration file (NeFsServer.exe.config) :
o TraceFile, TraceLevel, PrivateKeyFile, the sink provider (NefsServerChannelSinkProvider), the NeFsServerFormatterSinkProvider custom formatter and either SoapServerFormatterSinkProvider or BinaryServerFormatterSinkProvider. Either formatter can be used alternatively in this context if the system is run on local network.
- When initializing the sink provider object we collect the relevant information from the configuration file using a Dictionary object passed to the constructor as shown in the code fragment below.
The channel sink provider initializes the ServerChannelSink using
- When message is passed from the client the ServerChannelSink Processmessage method receives the message and applies the NeFsCryptoService provider methods to decrypt the message and pass it to the ServerFormatterSink Processmessage method that processes the message by invoking the relevant remote methods as is indicated by the transparent proxy object that resides in the client.
- Note that the order of message processing in the client side is the reverse of the order of message processing in the server side.
- To track the process more easily, a trace object is included. The relevant information of the transport headers, messages and the stream at each point of execution are dumped on the console screen for the server and on the output pane of Visual Studio for the client. Closely examining the output will provide more insight into the working mechanism of the underlying transport and message sinks discussed above.
- The Cryptography assembly implements four classes. These are:
NefsCryptoServiceProvider.cs: This class implements the encrypting and decrypting methods using the private/public keys passed to the methods as an argument. The process of encryption and decryption is briefly discussed below for the sake of clarity.
Encryption
- The private key (obtained from the configuration files) is passed to the object during initialization.
- The NeFsEncrypter method is called by the channel sinks with the stream of data to be encrypted and the public key as shown in the code fragment below.
- The NefsSecuritySigner signs the encoded data. It uses the private key to sign the Xml Document version of the data stream and returns an Xml Element.
- The signed data is further scrambled using randomly generated bytes obtained from .NETs PasswordDeriveBytes object using randomly generated array of bytes namely; secret key, iteration and ID. These bytes are formatted using RSAPKCS1KeyExchangeFormatter and the public key.
- The result is stored in tailored objects namely; NefsSignedData and NefsData structure.
- These wrapper objects will finally be serialized in a stream using .NETs XmlSerializer and returned to the caller channel.
Decryption
- The process of decryption is the reverse of the encryption process.
- We first de-serialize the NefsSignedData object from the Xmlized stream.
- Using the RSAPKCS1KeyExchangeDeformatter(prvKey) key exchange deformatter we de-format the keys used to scramble the Xml element. Note that we are using the private key of the encrypters public key that is used to format the key. (If the client and server private/public keys are out of sink we are in trouble. That is why we have the Key Store Manager in place. See below)
- Using the keys we unscramble the data.
- We pass the unscrambled data to the NefsSecurityVerifier method to extract the public key and the signed data from NefsSignedData object.
- We verify data-key integrity and return the decrypted data stream to the caller channel sink.
- More hands on exercise and observation of the trace output, hopefully, will provide you with more insight into the coding and working mechanism of the crypto process.
- NefsSignature: is an internal class that provides signing encoded data and verifying decoded stream of data. (The role of this class is as explained above in the encryption and decryption processes).
- NeFsKeyManager: provides basic object serialization functionality. Using the SoapFormatter of .NET it serializes a hashtable object containing private/public key strings into a file stream. In addition to this it provides Add, Remove and other methods. The class along with the corresponding Key Manger interface at GUI level allows the secret keys of the client and the server to be in sink at all time.
- NefsScrambler: Provides a simple message scrambling facility. The scrambling is based on cryptic dynamic secret parameters that are generated at the client and server side and exchanged across using the .NET Key exchange formatter and de-formatter classes mentioned above.
Data Access Assemblies
Design Issue
1. The design of the data access component addresses the following three issues:
- Mapping the underlying objects of the relational database to the application objects;
- Optimization of performance within the remote data access context; and
- Provide a framework that will nullify or at least minimize the change in the code and the underlying data access structure due to post-release or enhancement changes in the underlying database.
2. C# and the .NET framework include a number of complex architectures that allow us to integrate data access components to the application in either disconnected or connected fashion to the underlying database.
3. However, I found it a bit daunting when it comes to mapping relational database structure to the more complex data access facilities provided by .NET framework. (HOPE I AM NOT ALONE ON THIS!!!!!!)
4. The option that I followed is depicted in figure 2.
Figure 2: Data Access Component
5. The implementation classes that are mapped to the data tables inherit from the NefsDataEntity abstract class and implement the NeFsDataAttribute class custom attributes. (See Code fragment below)
6. The implementation classes simply get the attribute values and delegate the method execution to the static methods of NeFsDataAccess class.
7. The NeFsDataAccess object using GetAttributes method resolves the arguments of the methods (Load, Update, Save, create Relation etc) and returns that to the caller implementation class which in turn passes that to the client.
8. Note that the NefsDataAccess methods are completely separated from the corresponding implementation class. The implementation classes, including the client side, can grow or shrink without affecting the underlying data access methods. A close observation of the implementation classes (both client and server side) reveals that the routines are repetitive and can be auto-generated using a custom auto tool (something to think about if one tries to follow this sort of Data Access Design). Or on the client side, these classes can be generated using Class Factory and/or Adaptor type patterns that will recreate the classes dynamically at run time. This is particularly, important in a large scale trading application where the model association framework relies on the Security Valuation Data retrieved from various tables to evaluate a given instrument. Linking the Auto generator with the SQL script, in particular, serves to respond to database changes more effectively and efficiently and the current design exactly enable us to achieve this.
9. Note that I have deliberately separated the SQL statements and included them within the database realm as is shown with the green shaded area in Figure 2. This not only makes the code very readable but also enables us to design a custom Data Access implementation class generator solely depending on the SQL script, there by considerably minimizing development cost associated with database enhancement or change.
10. In addition to the easy maintenance cost that this design provides, it also addresses performance issue. As might be appreciated, accessing data remotely costs the performance of the application very heavily.
11. To overcome this, the application is only priced a one off performance cost during initialization. The rest of the interaction between user interface and the database for loading, updating, saving and deleting is handled using ADO.NETs DataSet object and a series of worker threads (see code fragment below) that pass the information back and forth with out affecting the performance in the background.
The FinAnalytics Assembly
1. Whether we want to access calculation agents (business methods and logics) remotely or do it locally within a Peer2Peer networked system is something that I have paid no attention in the current demo application. And I go along with both options depending on the circumstances. However, it should be noted that there is some performance cost incurred during computation within the current design. One-way out of this is to have Serializable objects that will be passed to the client by Value. In the present context I have put them as objects passed by reference just for the fun of it. I agree if someone argues that it should have been better if I should have done it otherwise!!!.
2. Assuming that you go along with me on this point, the analytics is modeled using the following framework.
3. The data input/output delivery objects are separated from the actual calculating agents or objects. These objects are in NeFsSerializableLogicalObjects and use CallContext objects to pass data from client to server and vise versa.
4. The NeFsUnderLyingParameters object is responsible for passing the required input parameters of the Trinomial and Black-Sholes option Pricing models. Of these two the trinomial model is a bit complex for coding and requires some mathematical background to understand the code. If you are up to it try to look the code in BuildNodes.cs and NodeParameters.cs files to see hot nodes of the tree are build.
5. The Monte Carlo simulation model is a simple multiplicative recursive model that is used to approximate the lognormal distribution of CALL/PUT premiums given the initial inputs. (Those who are interested to look into the models look the reference book sited at the end of this article.)
Namespaces, Files and Installation Guide
1. The demo application is organized into the following namespaces:
| NefsSystem | The root namespace, it includes the NefsKeyComparer ,NeFsUtility and NeFsRemoteObject |
| NefsSystem.Analytics | This namespace includes the basic analytic models. The classes, interfaces and structs included under this namespace are: BlackSholesStdOption, BuildNodes, NodeParameters, NodeUnderlyingPrice,NodeData, TriNodeCollection, IoptionPricing, NeFsValuation, Simulation, TrinomialstdOptionsq |
| NefsSystem.Cryptography | This is the crypto service provider name space. It includes: NefsCryptoServiceProvider, NefsSignedData, NefsData, NefsSignature, INeFsKeyManager, NeFsKeyManager, NefsScramblerq |
| NefsSystem.DataAccess | This includes the base data access and database connection facility. It includes: DataAccessParameters, NeFsDataEntity, NeFsDataAccess, NeFsDataAttribute, NeFsDataSet, UserAccess, Trade, TradeDetails, Security, UnderLyingParameters, CashFlow, Curve, CurveData, Dividend, DividendData, NeFsStatusList, NefsCurrencyList |
| NefsSystem.Remoting | Includes two namespaces for formatter and secured channel sink classes. In addition at its root level it includes: IneFsRemoteObject. This class serves as a metadata or remote object interface used by the client. |
| NefsSystem.Remoting.MessageFormatter | This namespace includes the classes that are responsible for providing message formatter sinks. It includes: NeFsClientFormatterSink, NeFsClientSink, NeFsClientFormatterSinkProvider, NeFsServerFormatterSink, NeFsServerFormatterSinkProvider |
| NefsSystem.Remoting.SecuredChannel | This namespace includes the channel sink provider classes that are secured with the crypto service provider. It includes: NefsClientChannelSink, NefsClientChannelSinkProvider, NefsServerChannelSink, NefsServerChannelSinkProvider |
| NefsSystem.SerializableLogicalObjects | Serializable objects together with those that are passed back and forth between client/server using CallContext objects are included under this namespace: It includes: Connection, NeFsUnderLyingParameters, SimulatedValues, ColumnInformation |
| NefsSystem.Trace | This namespace includes facilities related to tracing runtime execution of the underlying methods in the system. It includes: NefsTrace, NefsTrackingHandler |
| NefsSystem.ClientDataComponent | This is client side data access method provider. It includes: NeFsDataLink, UserDataAccess, CurrencyList, TradeStatus, Dividend, DividendData, Curve, CurveData, CashFlow, UnderlyingParameters, Security, TradeDetails, Trade |
| NefsSystem.WindowsClient | The user interface includes 13 forms and the following helper classes NeFsEventArgs, NeFsEventMonitor, NeFsEventManger, BuildMenu, MenuArgs,and NeFsGraphicHelper |
2. File Organization and Installation issues: I have included source code files only. The zipped files include folder information of source files. Following the direction detailed in the table below build the projects using the visual studio and ADD the relevant files from each folders. (I have used the released version of VS, the source code was not tried on the beta releases, but if you try it, it will work though it might require you to fix code here and there.)
| Project Name | Project Type | Project Files | Required External Files |
Referenced Modules (You need to compile the indicated modules before compiling the corresponding this module) or get the modules if they are refrenced .NET modules |
| NeFsServer | Console Application | NefsServer.cs | privateKey.xml, NeFsServer.exe.config | NefsAnalytics, NefsCryptography, NefsDataAccess, NeFsRemoteObject, NefsRemoting, NeFsSerializable LogicalObjects, NefsTrace, NefsUtility, System.dll, System.Data.dll, System.Runtime .Remoting, System.Xml.dll |
| NeFsWindowsClient | Windows Application | NeFsAddNewUser.cs, NeFsClientKeyManager.cs, NeFsConnection.cs, NeFsCurveDataBuilder.cs, NeFsDividendDataBuilder.cs, NeFsLogin.cs, NeFsMainWindow.cs, NeFsModuleManager.cs, NeFsNewTrade.cs, NeFsSimuResults.cs, NeFsSimulate.cs NeFsSimuResults.cs, NeFsTradeCapture.cs NeFsTradeScreen.cs NeFsURIBrowser.cs, NeFsWinUtilities.cs |
NeFsWindowsClient. exe.config, KeyStore.xml, privateKey.xml (both previous files are automatically created for you just provide the name and path (whatever name and path you like) in the configuration file in its relevant section. System.Security, System.Windows .Forms.dll, System.Xml.dll |
eFsClientData AccessComponent, NefsCryptography, NefsRemoting, NeFsSerializable LogicalObjects, NefsTrace, NefsUtility, System.dll, System.Configuration .Install.dll, System.Data.dll, System.Drawing.dll, System.Runtime.Remoting, |
| NeFsAnalytics |
Class Library | AnalyticsUtility.cs, BlackSholesStdOption.cs, BuildNodes.cs, NodeParameter.cs, Option.cs, Simulation.cs, TrinomialstdOptions.cs |
NeFsSerializable LogicalObjects, System.dll, System.Data.dll, System.Xml.dll | |
| NeFsClientData AccessComponent |
Class Library | ClientDataComponent.cs | NefsDataAccess.cs (This file is generated using soapsuds from NefsDataAccess assembly generate the file using the relevant command in DOS and copy the file to this projects directory. | NefsRemoting, NeFsSerializable LogicalObjects, System.dll, System.Xml.dll, System.Data.dll |
| eFsCryptography | Class Library | NefsCryptoServiceProvider.cs, NefsKeyManager.cs, NefsScrambler.cs |
NefsDataAccess.cs, NefsDataAttributes.cs, NeFsDbImplementation.cs |
NefsTrace, NefsUtility, System.dll, System.Data.dll, System.Runtime .Serialization .Formatters .Soap, System.Security, System.Xml.dll |
| NeFsDataAccess | Class Library | NeFsSerializable LogicalObjects, NefsTrace, System.dll, System.Data.dll, System.Xml.dll | ||
| NeFsRemoteObject | Class Library | NefsRemoteObject.cs | NefsAnalytics, NefsDataAccess, NefsRemoting, NeFsSerializable LogicalObjects, System.dll, System.Data.dll, System.Xml.dll | |
| NeFsRemoting | Class Library | NefsClientChannelSink.cs, NefsClientChannelSinkProvider.cs, NeFsClientFormatterSink.cs, NeFsClientFormatterSinkProvider.cs, NefsServerChannelSink.cs, NefsServerChannelSinkProvider.cs, NeFsServerFormatterSinkProvider.cs, NeFsServerFormatterSink.cs NefsTransparentProxy.cs, |
NefsAnalytics, NefsCryptography, NefsDataAccess, NeFsSerializable LogicalObjects, NefsTrace, System.dll, System.Data.dll, System.Runtime .Remoting, System.Runtime .Serialization. Formatters.Soap, System.Security, System.Xml.dll | |
| NeFsSerializable LogicalObjects |
Class Library | NeFsDAccessSerializable LogicalObjects.cs, NeFsOptParSerializableLogical.cs, NeFsSimSerializableLogicalObjects.cs, NefsTriSerializableLogicalObjects.cs |
System.dll, System.Data.dll, System.Xml.dll | |
| NeFsTrace | Class Library | NefsTracer.cs, NefsTracer.cs |
NefsUtility, System.dll, System.Data.dll, System.Runtime .Remoting, System.Xml | |
| NeFsUtility | Class Library | NefsHelper.cs | System.dll, System.Data.dll, System.Xml.dll |
3. SQL script: the SQL script enables you to create the MS SQL Server database ( I have used MS SQL Server 2000 to create the script and the database). The file is FinNet.sql.
Running the Application
Once you downloaded the code and managed to build the projects, run the application. Note that we need the following modules and files at the client and server side.
Client-side
- NeFsClientDataAccessComponent.dll;
- NefsCryptography.dll;
- NefsCryptography.dll;
- NeFsSerializableLogicalObjects.dll;
- NefsTrace.dll;
- NefsUtility.dll;
The following are files that you either generate or use those provided with the code (Note that you need to modify the configuration file if you changed the names of the files and the corresponding path).
- KeyStore.xml;
- NeFsWindowsClient.exe.config;
- privateKey.xml (this file is aut generated by the KeyStore manager that reconciles the public key with the private key. Copy this file to the server side folder or wherever you want to put it. But do not forget to modify the server configuration file accordingly.
Server side
- NeFsClientDataAccessComponent.dll;
- NefsCryptography.dll;
- NefsCryptography.dll;
- NeFsSerializableLogicalObjects.dll;
- NefsTrace.dll;
- NefsUtility.dll;
- NefsAnalytics.dll;
- NefsDataAccess.dll;
- NeFsRemoteObject.dll;
The following are files that you either generate or use those provided with the code (Note that you need to modify the configuration file if you changed the names of the files and the corresponding path).
- NeFsServer.exe.config;
- privateKey.xml (this is copied from the client side. Well it should be the server that need to manage the keys but the since it is a Console application I have dumped it in the client side.
Once the files are in place run the server first followed by the client either in debug or release mode. Follow the self-explanatory user interface of the application to price options and run simulation. Note that some of the menus are not implemented in the current version. Try to complete the incomplete menus and come up with a complete version of the demo application. This will give you a chance to pick-up more out of the code and learn more about C# and .NET. I HOPE YOU WILL AGREE WITH ME .
Conclusions
In this article and the corresponding code I have tried to explain the possible ways of implementing the C# and .NET framework within Networked application context. There are a number of issues that I have addressed. But to keep the complexity of the presentation as simple as possible I have avoided going through the code step by step. In stead I have highlighted the most significant part of the demo that I think will help others in their effort to do similar type of exercise.
Though the exercise has achieved what it is set out to achieve, still there is one issue that I have not gone through in this article. This is the issue of reusing or deploying the Remote, Data Access, and Analytics components and objects for Web based client. In my next article I will post how this could be achieved with minimal change to the existing components.
Mohammad HelalPosted Apr 1, 2013, 1:29 PM
THNKS, this demo is nice.. i love it
Anurag SarkarPosted Jan 14, 2013, 2:12 PM
There should be option to vote also. Nice Thanks for sharing (*)+.