Project_Report (BARC-Jerin)_final

Internship Report
On
Dynamic Query Processing using C#
Prepared by,
Jerin J. John
Year: 2015-2016
K. J. Somaiya College of Engineering
Vidyavihar, Mumbai - 77

CERTIFICATE
This is to certify that Jerin J. John, 2nd year student of Computer
Engineering Dept., K. J. Somaiya College of Engineering has undergone
summer internship on the project Dynamic Query Processing using C#, from
23-05-2016 to 30-06-2016 at Nuclear Information Systems Section,
Electronics Division, Bhabha Atomic Research Centre, Trombay under
my guidance and supervision.
Smt. Padmini S.
Project Guide

ACKNOWLEDGEMENT
I am highly indebtedto Smt. Padmini S., SO/H+, NISS, ED, BARC for taking out
time from her busy schedule and providing me with continuous guidance,
motivation and support throughout the project activity.
I would like to take this opportunity to express my deep gratitude to all staff of
NISS, BARC, for their valuable support and suggestions in my project work.
Withouttheiradvice and help, it would have been very difficultto complete this
work.

INDEX
 Aim of the Project
 Introduction
 Language and Concepts
o Delegates
o Lambda Expressions
o LINQ
o Dynamic Code Compilation
 Project Program Code
 Screenshots – Program Output

Aim of the project
Develop a program in C# to compile and execute a Linq query at runtime, from a text
file or text input, on data stored in any Xml file.
Introduction
The project focuses on the basic idea of dynamically querying data from a C# program
using the concept of LINQ and dynamic code compilation.
A query is a request for information from a database. Depending on the database
management system, there are multiple query languages which can be used to perform
operations on the data.
The most commonly used format is called Structured Query Language [SQL] which
provides the platform to access a database and perform operations like insertion,
updation, deletion, etc.
C# language supports a technology called LINQ, which stands for Language Integrated
Query, which brings all the features and flexibility of SQL right into the programming
language. LINQ can be used to query data across multiple platforms from XML to
databases. LINQ combined with other C# language concepts like lambdas and
delegates, defines a highly efficient way to create and execute queries from programs.
In this project, usage of Linq is demonstrated on data stored in XML files. The data is
fetched from the XML file and is stored in the form of a generic List<>, which is then
queried using Linq.
The project scope has been expanded to support dynamic querying, i.e., obtain a query
either from text file or user input at runtime and operate it on the data.
Dynamic Querying is incorporated by using the Code Compilation assemblies provided
by C#, which enables programs to create, compile and execute programs from within
themselves. This concept is utilized to create an internal program consisting of a
method, which dynamically processes the query
on the datalist and returns its resultset.
The execution flow of the program involves:-
 Reading data and queries from their respective files.
 Initializing the dynamic code compiler and setting its respective parameters.
 Defining the internal program and method which is responsible for execution of
the query.
 Compiling the source code and obtaining an assembly.
 Creating an instance and calling the internal method to execute the query.
Future Scope
The scope of this program can be expanded to dynamically query data stored across
multiple platforms like databases, DOMs, etc and hence can be applied to several
systems.

Language and Concepts
C# is a modern, general-purpose, multi-paradigm programming language
encompassing strong typing, imperative,declarative, functional, generic, object-
oriented (class-based), and component-oriented programming disciplines developed
and maintained by Microsoft and approved by European Computer Manufacturers
Association (ECMA) and International Standards Organization (ISO).
C# is designed for Common Language Infrastructure (CLI), which consists of the
executable code and runtime environment that allows use of various high-level
languages on different computer platforms and architectures.
C# constructs closely follow traditional high-level languages, C and C++ being an
object-oriented programming language. It has strong resemblance with Java, and has
numerous strong programming features that make it endearing to a number of
programmers worldwide.
Following is the list of few important features of C#:
 Boolean Conditions
 Automatic Garbage Collection
 Standard Library
 Assembly Versioning
 Properties and Events
 Delegates and Events Management
 Easy-to-use Generics
 Indexers
 Conditional Compilation
 Simple Multithreading
 LINQ and Lambda Expressions
 Integration with Windows

Delegates
C# delegates are similar to pointers to functions, in C or C++. A delegate is a reference
type variable that holds the reference to a method with a particular parameter list and
return type. The reference can be changed at runtime. When you instantiate a
delegate, you can associate its instance with any method with a compatible signature
and return type. You can invoke (or call) the method through the delegate instance.
Delegates are used to pass methods as arguments to other methods. Event handlers
are nothing more than methods that are invoked through delegates. Any method from
any accessible class or struct that matches the delegate type can be assigned to the
delegate. The method can be either static or an instance method. This makes it possible
to programmatically change method calls, and also plug new code into existing classes.
Delegates are especially used for implementing events and the call-back methods. All
delegates are implicitly derived from the System.Delegate class.
Syntax for delegate declaration is:
delegate<return type><delegate-name><parameter list>
Lambda Expressions
A lambda expression is an anonymous function that you can use to create delegates or
expression tree types. By using lambda expressions, you can write local functions that
can be passed as arguments or returned as the value of function calls. Lambda
expressions are particularly helpful for writing LINQ query expressions.
To create a lambda expression, you specify input parameters (if any) on the left side of
the lambda operator =>, and you put the expression or statement block on the other
side. For example, the lambda expression x => x * x specifies a parameter that’s
named x and returns the value of x squared.
Lambdas are used in method-based LINQ queries as arguments to standard query
operator methods such as Where<TSource>.
 Expression Lambdas
A lambda expression with an expression on the right side of the => operator is called
an expression lambda. Expression lambdas are used extensively in the construction of
Expression Trees (C# and Visual Basic). An expression lambda returns the result of the
expression and takes the following basic form:
(input parameters) =>expression Eg : (x, y) => x == y

 Statement Lambdas
A statement lambda resembles an expression lambda except that the statement(s) is
enclosed in braces:
(input parameters) => {statement;}
The body of a statement lambda can consist of any number of statements; however, in
practice there are typically no more than two or three.

LINQ
Language Integrated Query (LINQ, pronounced "link") is a Microsoft .NET Framework
component that adds native data querying capabilities to .NET languages, although
ports exist for Java, PHP, JavaScript and ActionScript.
LINQ (Language Integrated Query) allows writing queries even without the knowledge
of query languages like SQL, XML etc. LINQ queries can be written for diverse data
types.
Language-integrated query allows query expressions to benefit from the rich metadata,
compile-time syntax checking, static typing and IntelliSense that was previously
available only to imperative code. Language-integrated query also allows a single
general purpose declarative query facility to be applied to all in-memory information,
not just information from external sources.
LINQ Architecture in .NET
LINQ has a 3-layered architecture in which the uppermost layer consists of the
language extensions and the bottom layer consists of data sources that are typically
objects implementing IEnumerable<T> or IQueryable<T> generic interfaces. The
architecture is shown below in Figure.

Advantages of LINQ
LINQ offers a host of advantages and among them the foremost is its powerful
expressiveness which enables developers to express declaratively. Some of the other
advantages of LINQ are given below.
 LINQ offers syntax highlighting that proves helpful to find out mistakes during
design time.
 LINQ offers IntelliSense which means writing more accurate queries easily.
 Writing codes is quite faster in LINQ and thus development time also gets
reduced significantly.
 LINQ makes easy debugging due to its integration in the C# language.
 Viewing relationship between two tables is easy with LINQ due to its hierarchical
feature and this enables composing queries joining multiple tables in less time.
 LINQ allows usage of a single LINQ syntax while querying many diverse data
sources and this is mainly because of its unitive foundation.
 LINQ is extensible that means it is possible to use knowledge of LINQ to
querying new data source types.
 LINQ offers the facility of joining several data sources in a single query as well as
breaking complex problems into a set of short queries easy to debug.
 LINQ offers easy transformation for conversion of one data type to another like
transforming SQL data to XML data.
Dynamic Code Compilation
Dynamic code execution is a powerful feature that allows applications to be extended
with code that is not compiled into the application. Users can customize applications
and developers can dynamically update code easily. The .Net framework introduces
classes that simplifies the tasks by wrapping the details of the process in an easy to use
interface that only requires a few lines of code.
.Net provides powerful access to the IL code generation process through the
System.CodeDom.Compiler and Microsoft.CSharp and Microsoft.VisualBasic
namespaces. In these namespaces you find the tools that allow you to compile an
assembly either to disk or into memory. You also need the Reflection namespace as it
contains the tools to invoke an object and its methods once you've compiled the object.
The process to execute this code dynamically involves the following steps:
 Create or read in the code that is to be executed as a string or from file.

 Wrap the code into fully functional assembly source code, which includes
namespace references (using commands), a namespace and a class that is to be
invoked.
 Compile the source code into an assembly.
 Check for errors on compilation.
 Use the assembly reference to create an instance of the object.
 Call the specified method on the instance reference returned using Reflection.
 Handle any return value from the method call by casting into the proper type.
The CodeDOM (Code Document Object Model) provides types that represent many
common types of source code elements. You can design a program that builds a source
code model using CodeDOM elements to assemble an object graph. This object graph
can be rendered as source code using a CodeDOM code generator for a supported
programming language. The CodeDOM can also be used to compile source code into a
binary assembly.
Project Code ( C# - program.cs )
using System;
using System.Collections;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Xml.Linq;
using System.Dynamic;
using System.CodeDom;
using System.CodeDom.Compiler;
using System.Linq.Dynamic;
using Microsoft.CSharp;
namespace DynQueryProcessor
{

class Program
{
// .. Initialize dynamic code generation and compilation objects
static CSharpCodeProvider cdp = new CSharpCodeProvider();
static CodeCompileUnit ccu = new CodeCompileUnit();
static CompilerParameters cparams = new CompilerParameters();
static CodeEntryPointMethod start = new CodeEntryPointMethod();
static CodeNamespace ns = new CodeNamespace("Internal");
static string filepath = @"F:Jerin'JerinCodetest5";
static void Main( string[] args )
{
#region Initialisation //Section
CodeTypeDeclaration m_class = new CodeTypeDeclaration("Program");
// .. Importing assemblies required for internal code
ccu.ReferencedAssemblies.Add("System.dll");
ccu.ReferencedAssemblies.Add("System.Core.dll");
ns.Imports.Add(new CodeNamespaceImport("System"));
ns.Imports.Add(new CodeNamespaceImport("System.Collections.Generic"));
ns.Imports.Add(new CodeNamespaceImport("System.Linq"));
ccu.Namespaces.Add(ns);
Console.ForegroundColor = ConsoleColor.Green;
// .. Load the XML document containing data to be queried
XDocument xd = XDocument.Load(@"F:Jerin'JerinXmltest.xml");
XElement xe1 = XElement.Load(@"F:Jerin'JerinXmltest.xml");
// .. Load the text file containing query to be executed
string queryfile = @"F:Jerin'JerinQueryTest1.txt";
string query = QueryReader(queryfile);

//string modq = QueryIsolator(query);
Console.Write(" Enter name of field you wish to query
(Student/Employee/Department) : ");
List<dynamic> datalist = XmlListGen(xd , Console.ReadLine());
start.Comments.Add(new CodeCommentStatement("methodregion"));
m_class.Members.Add(start);
ns.Types.Add(m_class);
#endregion
#region Experiment //Section (LinQ)
//var elist = xe1.Elements();
//var resultset = datalist.Where(dept => dept.DName.Contains
("Computer")).Select(dept => dept.Name);
//XmlWriter xw1 = xe1.CreateWriter();
//XmlReader xr1 = xe1.CreateReader();
//xr1.ReadStartElement();
//String s1 = xr1.ReadOuterXml();
//Console.WriteLine(s1);
//XmlElement x1 = xd.DocumentElement;
//var slist = elist.Where().Select( file => file );
//var slist1 = elist.Descendants().Where( e => e.Value.Contains( sval ) ).Select(
e => e );
//foreach( var item in resultset )
//{
// Console.WriteLine( item );
//}
#endregion
// .. Add required assemblies to the compiler parameters
cparams.ReferencedAssemblies.Add( "System.dll" );
cparams.ReferencedAssemblies.Add( "System.Core.dll" );

cparams.ReferencedAssemblies.Add( "Microsoft.CSharp.dll");
cparams.ReferencedAssemblies.Add( "System.Data.dll");
cparams.ReferencedAssemblies.Add( "System.Xml.dll");
cparams.ReferencedAssemblies.Add( "System.Xml.Linq.dll" );
cparams.GenerateExecutable = false;
cparams.GenerateInMemory = true;
cparams.TreatWarningsAsErrors = false;
cparams.CompilerOptions = "/optimize";
string codefile = CodeFileGenerator(ccu);
// .. Code for internal method to execute the query
string methodcode = "nt public List<dynamic> QueryFire(List<dynamic> data)
nt{" +
"ntt try {" +
"nttt var result = "
+query+".Cast<dynamic>().ToList();" +
"nttt return result ;" +
"ntt }ntt catch(Exception ex) " +
"ntt { nttt return new List<dynamic>
{ex.Message + ex.StackTrace}; ntt } nt } nn";
// .. Replacing comment in internal program with above query method
File.WriteAllText(codefile, System.Text.RegularExpressions.Regex.
Replace(File.ReadAllText(codefile), "// methodregion", methodcode));
using (StreamReader sr = new StreamReader(filepath))
{
while (!sr.EndOfStream)
Console.WriteLine(sr.ReadLine()); }
Console.WriteLine("-------------------------------------------------- nnt Query Output :-
n");
#region Execution // Section
// .. Obtain an assembly compiled from the codefile

return query;
}
// .. Method to return a List<dynamic> from XML data to be queried
static List<dynamic> XmlListGen(XDocument xd , string searchfield)
{
List<dynamic> datalist = new List<dynamic>();
if (searchfield == "Employee")
{
var elist = xd.Descendants("Employee").Select(s => s);
foreach (var item in elist)
{
dynamic emp = new ExpandoObject();
emp.Ename = item.Element("Ename").Value;
emp.Salary = item.Element("Salary").Value;
emp.DName = item.Element("DName").Value;
emp.EmpID = item.Element("EmpID").Value;
datalist.Add(emp);
}
{
Console.WriteLine(item.Name + item.Value);
}
}
else if (searchfield == "Student")
{
var elist = xd.Descendants("Student").Select(s => s);
{
dynamic student = new ExpandoObject();
student.Name = item.Element("Name").Value;
student.Ssn = item.Element("Ssn").Value;
student.DName = item.Element("DName").Value;
datalist.Add(student);
}

{
}
}
else
{
var elist = xd.Descendants("Department").Select(s => s);
{
dynamic dept = new ExpandoObject();
dept.Intake = item.Element("Intake").Value;
dept.DName = item.Element("DName").Value;
datalist.Add(dept);
}
{
}
}
return datalist;
}
static string QueryIsolator(string query)
{
Dictionary<string , string> qsegments = new Dictionary<string , string>();
string nminus = query.Substring( query.IndexOf( '.' ) );
string descendantq = null;
qsegments.Add( "nminus" , nminus);
string listname = query.Remove(query.IndexOf
(qsegments.Where( q => q.Key == "nminus" ).Select( q => q.Value).Single()));
qsegments.Add( "listname" , listname );
if (query.Contains("Descendants"))

{
descendantq = nminus.Substring(nminus.IndexOf("Descendants"));
descendantq = descendantq.Remove(descendantq.IndexOf(")"));
descendantq = descendantq.Replace("Descendants(", "");
qsegments.Add("desc", descendantq);
}
if ( query.Contains( "Where" ))
{
string whereq = nminus.Substring( nminus.IndexOf( "Where" ) );
whereq = whereq.Remove( whereq.IndexOf( ")" ) );
whereq = whereq.Replace( "Where(" , "" );
qsegments.Add( "where" , whereq );
}
if( query.Contains( "Select" ) )
{
string selectq = nminus.Substring( nminus.IndexOf( "Select" ) );
selectq = selectq.Remove( selectq.IndexOf( ")" ) );
selectq = selectq.Replace( "Select(" , "" );
qsegments.Add( "select" , selectq );
}
Console.WriteLine(nminus);
return descendantq;
}
// .. Method to generate a code file from requested parameters
static string CodeFileGenerator(CodeCompileUnit ccu)
{
CSharpCodeProvider provider = new CSharpCodeProvider();
if (provider.FileExtension[0] != '.')
{
filepath += ".";
}
filepath += provider.FileExtension;
using (StreamWriter sw = new StreamWriter(filepath, false))

{
IndentedTextWriter itw = new IndentedTextWriter(sw);
provider.GenerateCodeFromCompileUnit(ccu, itw, new
CodeGeneratorOptions());
}
return filepath;
}
}
}
Program Output Screenshots:
 Field to be queried.

 Dynamically generated internal program containing method - QueryFire()
to process the query.

GraphicalUser Interface implementation of Project
 Select location of XML File and verify using file preview.

 Select location of Query File, select domain to operate upon
and verify using file preview.

 Enter Query into text field by choosing “ Enter Query “ option
and select domain to operate upon.

Project_Report (BARC-Jerin)_final

Recommended

More Related Content

What's hot (20)

Viewers also liked (17)

Similar to Project_Report (BARC-Jerin)_final (20)

Project_Report (BARC-Jerin)_final