HTML parsing

In a very small personal project I needed to parse an HTML page. I thought that HTML = XML and so I tried to load the response string into an XDocument; but I was too optimistic. I got a lot of errors from the XDocument.Parse() method. I read on the internet that it is only possible to do this if the web page is XHTML and follows all the standards. But there are a lot of web pages out there which never heard nothing about standards. A good example are pages from Microsoft. This page of the Sysinternals Suite for example produces 53 errors and 14 warnings when validating it with the W3C validator!!

There is a solution to the HTML parsing problem for C#. There exists a project called HTML Agility Pack on CodePlexx which provides a mechanism to parse an HTML string and to navigate it like ian XmlDocument. It works great but it does not provide support for LINQ queries which is very nasty if you are accustomed to it (like me :-))
But also for this exists a solution. Since the source code of the HTML Agility Pack is open for everyone Vijay Santhanam created the ToXDocument() method, which converts the HtmlDocument to XDocument. This is then queryable with LINQ to XML. The post about this and a link to his project can be found here.

It would be nice if they would add direct LINQ support to their HtmlDocument class but maybe they will do so in the future...

XSD element order

Recently I faced the problem that I had to create an XSD file to validate an XML file. The problem was that a requirement was that the elements could be in any order.
First I had something like:

<xs:sequence>
<xs:element name="customer" />
<xs:element name="order" />
</xs:sequence>

which validates correctly:

<root>
<customer />
<customer />
<customer />
<order />
<order />
<order />
</root>

but I needed to validate something like:

<root>
<order />
<customer />
<customer />
<order />
<customer />
<order />
</root>

After some research I finally found the solution.
instead of sequence You have to use choice with the min and max occurs attribute set:

<xs:sequence>
</xs:sequence>

is replaced with

<xs:choice minOccurs="0" maxOccurs="unbounded">
</xs:choice>

Bulk insert of data

Recently I faced the problem of inserting a big amount of data into the database. The point is that multiple tables are affected which are connected with foreign keys. After some research I found OpenXML as one possibility. You create a stored procedure where you read your XML string into an "XmlDocument" which you can then query by using XPath and inserting, updating or deleting the tables occordingly.

The other possibility I found is SQLXML which is now available in version 4.1. It is delivered with SQL Server but not with the Express version. If you need it you can download it from here. SQLXML allows you to bulk insert data from an XML file by using a so called annotated XSD schema where the mapping is defined. It is not very difficult to define the schema, only recursion and special cases are a little bit difficult to handle. I was not able to make recursion work with identity values coming from the database. Anyway; I did some performance tests where I had a structure like:
Table A
Table B has a foreign key to Table A
Table C has a foreign key to Table B
which corresponds to the following XML structure:
<root>
<a>
<b>
<c></c>
</b>
</a>
</root>

I created an XML file that contained 100 A's; every A contained 100 B's and every B contained 100 C's. This gives a total of 100 A's, 10.000 B's and 1.000.000 C's. To insert all them and updating the foreign keys it took 84 seconds. This is quite fast for such an amount of data. I think that this is the fastest way to bulk insert data into SQL Server (if I'm wrong You are welcome to coment on this).
If you want to use it with .Net add a reference to "Microsoft SQL Server Bulk Upload" or something like that.
I don't remember the exact name but as soon as I find the link of the code sample I will post it here.

The problem is that SQLXML cannot be used for updates. There exist so called updategrams or diffgrams but I didn't try them.

Extension Methods in C#

There are some new features in C# 3 which are introduced to make LINQ possible.
One of them are extension methods which I wanted to try out. Extension methods are static methods which can be applied to any object type which is specified as "special parameter". For example if you want to extend the type string with the method "Greet" you write something like this:

public static void Greet(this String s)
{
Console.WriteLine("Hello " +  s);
}

Then you can call it like:

string name = "Tom";
name.Greet();

This will produce the following on the console: "Hello Tom"

I played a bit with this extension mehtods and came up with the following extensions for object which I believe can be helpful.
Assume the following object:

Customer testCustomer = new Customer() { Name = "John", Age = 35, LastName = "Smith" };

PrintConsole() -> prints the string representation of an object to the console
Code:

testCustomer.PrintConsole();

Produces:
"ExtensionMethodTest.Customer"

PrintMessageBox() -> shows the string representation of an object in a message box
Code:

testCustomer.PrintMessageBox();

Produces:


PrintPropertiesConsole() -> Prints all public non inherited properties and their values of the object to the console
Code:

testCustomer.PrintPropertiesConsole();

Produces:
Properties of 'ExtensionMethodTest.Customer'
Name: John
LastName: Smith
Age: 35

PrintPropertiesMessageBox() -> Shows all public non inherited properties and their values of the object in a message box
Code:

testCustomer.PrintPropertiesMessageBox();

Produces:

string ToXml() -> Serializes the object to XML and returns the XML string
Code:

testCustomer.ToXml();

Produces:

John
Smith
35

ToXml(Stream stream) -> Serializes the object to XML and writes it to the stream
Code:

testCustomer.ToXml(myStream);

ToXml(string fileName) -> Serializes the object to XML and writes the XML string to the file specified.
Code:

testCustomer.ToXml("c://test.xml);

If you are interested in my project you can download it from here.
Or you can simply download the Extensions.dll from here and import it into your project.

All extension methods are in the namespace System. I know that this is not the way how to do it but so you don't have to add a using directive because System is by default present.

XPath

For them of you which do not know what XPath is; it is a very powerful query language for XML data. But it can also more than querying data; it can also compute values, find the max or min of an attribute and even produce a different output format. For example it is possible to write HTML code with XML data. C# provides different classes to execute XPath queries on XML files which are located in the "System.Xml.Xpath" namespace.

The problem, at least for me, is that I do not use XPath so often as that I always remember the sometimes tricky XPath-Syntax. Here I found it very helpful to have a graphical tool where I can type in my query and see the results of the query. I found such a tool which is freeware; Sketch Path. It is a very powerful tool which not only shows you the result of a query but also assists you when writing a query by providing common operations and selectors.

I can only encourage you to try XPath when working with XML data because it really makes it easier to get the data you want. Especially if you use XML files instead of a database. XPath can be seen as the SQL for XML files.

Here there is a screenshot of Sketch Path where I searched in a list of customers for all customers which are older than 50. The query is quite simple:
//Customer[Age>50]


Another nice thing which is very helpful is that if you select a node then it shows you the path to this node which can help you creating your query:



Here you can find a tutorial on XPath Syntax.