Abdullah's blog on software

TFS vNext needs audit log reports

2011-12-23T00:43:00.000-08:00

Because of some governmental regulations, we need to report the administrative actions over permissions on TFS. Auditors frequently request such reports from us.

I requested the audit logs of administrative actions over permissions on TFS from Microsoft in the VS vNext Wish List.

We'll see what Microsoft thinks about adding audit reports to TFS in the next version. What do you think, is it worth?

TFS Case Study of Microsoft Turkiye

2011-12-22T23:30:00.000-08:00

Microsoft Turkiye examined our TFS implementation and published a case study document:
http://www.microsoft.com/turkiye/casestudies/tfkb_tfs.aspx

Unfortunately the document is prepared only for Turkish audience, it has no English translation.

Count lines of code in a database

2011-06-19T22:36:00.000-07:00

Here is a small script to count lines of code in a database:

select
    o.type_desc as ObjectType
    ,count(*) as [ObjectCount]
    ,sum(len(m.definition) - len(replace(m.definition, char(13), ''))) as [TotalLineCount]
from sys.objects o
inner join sys.sql_modules m on o.object_id = m.object_id
where o.[type] in ('P', 'TR', 'FN', 'IF', 'V', 'TF')
and o.name not like 'aspnet[_]%'
and o.name not like 'vw[_]aspnet[_]%'
group by o.type_desc
order by o.type_desc

Sign a .Net assembly without the source code

2011-05-30T01:32:00.000-07:00

Two different versions of an old .Net assembly caused some deployment problems. We wanted to deploy the assembly to the GAC but neither of the two versions were signed. We needed to sign the assemblies but we did not have the source code.

After a small search I found a great tool named ILMerge.

Main purpose of ILMerge is to combine more than one .Net assemblies into one assembly. The side benefit is that you can also sign the resulting assembly. It works while working with only one assembly too. This way you can sign an assembly without having the source code.

Sample command line:

ilmerge /t:library /keyfile:"c:\SomeKeyFile.snk" /out:"c:\signed\assembly.dll" "c:\unsigned\assembly.dll"

Entity framework class generator

2011-05-04T01:20:00.000-07:00

I wrote a sql script to generate entity framework POCO classes: https://gist.github.com/954916

Script considers primary and foreign keys including both "one to one" and "one to many" relations.

It also writes sql object descriptions saved as extended properties to the class and property comments of POCO classes.

It is working for MS SQL 2008 and 2008 R2. I did not try to run the script on other versions.

C#: Call DynamicMethod within another DynamicMethod

2010-09-06T00:21:00.000-07:00

Today I tried to call a dynamic method within another dynamic method.

First I built a dynamic method, m1dyn:

DynamicMethod m1dyn = new DynamicMethod("m1", typeof(string), new[] { typeof(string) });
var il = m1dyn.GetILGenerator();
il.Emit(OpCodes.Ldarg_0);
il.Emit(OpCodes.Ret);

Then I created a delegate and assigned it to a delegate variable named m1:

Func<string, string> m1 = (Func<string, string>)m1dyn.CreateDelegate(typeof(Func<string, string>));

Then I built a second dynamic method, m2dyn:

DynamicMethod m2dyn = new DynamicMethod("m1", typeof(string), new[] { typeof(string) });
var il2 = m2dyn.GetILGenerator();
il2.Emit(OpCodes.Ldarg_0);
il2.Emit(OpCodes.Call, m1.Method);

I simply emitted a code which calls the method info of the first delegate variable, m1.
When I ran the code, I got the following error at the last line above:
"MethodInfo must be a RuntimeMethodInfo. Parameter name: meth"

What I had to do was to emit a call instruction with the first dynamic method variable itself, not the resulting delegate:
il2.Emit(OpCodes.Call, m1dyn);

Complete code is below:

public static void Test()
{
DynamicMethod m1dyn = new DynamicMethod("m1", typeof(string), new[] { typeof(string) });
var il = m1dyn.GetILGenerator();
il.Emit(OpCodes.Ldarg_0);
il.Emit(OpCodes.Ret);
Func<string, string> m1 = (Func<string, string>)m1dyn.CreateDelegate(typeof(Func<string, string>));

DynamicMethod m2dyn = new DynamicMethod("m1", typeof(string), new[] { typeof(string) });
var il2 = m2dyn.GetILGenerator();
il2.Emit(OpCodes.Ldarg_0);
il2.Emit(OpCodes.Call, m1dyn);
il2.Emit(OpCodes.Ret);
Func<string, string> m2 = (Func<string, string>)m2dyn.CreateDelegate(typeof(Func<string, string>));

string a = m2("a");
Console.WriteLine(a);
}

WCF BasicHttpBinding and windows authentication on IIS 6.0

2010-02-09T01:26:00.000-08:00

Recently I tried to host a WCF service on IIS 6.0 with basic http binding.
I needed to authenticate users with windows security mode.

I configured the WCF binding like below in the web.config file:

<bindings>
<basicHttpBinding>
<binding name="BasicHttpWindowsBinding">
<security mode="TransportCredentialOnly">
<transport clientCredentialType="Windows" />
</security>
</binding>
</basicHttpBinding>
</bindings>

Although I configured IIS so that anonymous authentication is not allowed and windows authentication is required, I got this error when I opened the service page with internet explorer:

System.NotSupportedException: security settings for this service require windows authentication but it is not enabled for the iis application that hosts this service

Well, it was disappointing because both service configuration and the IIS configuration was configured for windows authentication mode.

After a little struggle I found that basicHttpBinding's windows authentication is not supported by IIS 6.0.
Instead security mode of the binding should be set to Ntlm and you have to allow anonymous authentication in the IIS site settings.

Here is the complete config file:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>

<system.web>
<compilation debug="true" />
<customErrors mode="Off" />
<authentication mode="Windows">
</authentication>
<authorization>
<allow users="somedomain\someuser"/>
<deny users="*"/>
</authorization>
</system.web>

<system.codedom>
<compilers>
<compiler language="c#;cs;csharp" extension=".cs" warningLevel="4" type="Microsoft.CSharp.CSharpCodeProvider, System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
<providerOption name="CompilerVersion" value="v3.5" />
<providerOption name="WarnAsError" value="false" />
</compiler>
</compilers>
</system.codedom>

<system.webServer>
<validation validateIntegratedModeConfiguration="false" />
<handlers>
<add name="SvcFile" path="*.svc" verb="*" type="System.ServiceModel.Activation.HttpHandler" resourceType="Unspecified" preCondition="integratedMode" />
</handlers>
<directoryBrowse enabled="true" showFlags="Date, Time, Size, Extension, LongDate" />
</system.webServer>

<system.serviceModel>
<bindings>
<basicHttpBinding>
<binding name="BasicHttpWindowsBinding">
<security mode="TransportCredentialOnly">
<transport clientCredentialType="Ntlm" />
</security>
</binding>
</basicHttpBinding>
</bindings>
<services>
<service behaviorConfiguration="SomeServiceBehavior" name="SomeAssembly.SomeService">
<endpoint binding="basicHttpBinding" bindingConfiguration="BasicHttpWindowsBinding" contract="SomeAssembly.ISomeService">
<identity>
<dns value="localhost" />
</identity>
</endpoint>
<endpoint address="mex" binding="mexHttpsBinding" contract="IMetadataExchange" />
<host>
<baseAddresses>
<add baseAddress="https://localhost/SomeAssembly/ISomeService.svc" />
</baseAddresses>
</host>
</service>
</services>
<behaviors>
<serviceBehaviors>
<behavior name="SomeServiceBehavior">
<serviceMetadata httpGetEnabled="True"/>
<serviceDebug includeExceptionDetailInFaults="True" />
</behavior>
</serviceBehaviors>
</behaviors>
</system.serviceModel>

</configuration>

No history entries were found for the item and version combination specified

2009-11-06T04:14:00.000-08:00

This is a message when I tried to view the history of a folder on TFS.

I googled the message and only a few exact matches came out. First two was useless. I was terrified for an instance, but this blog entry saved me :

http://team-foundation-server.blogspot.com/2007/07/no-history-entries-were-found-for-item.html

The problem was that I had no permissions on the folder although I was an administrator. I could never remember to check the securities.

When I checked it I saw that the folder was not inheriting permissions. After inheriting permissions I could view the history of the folder.

Migrating from VSS to TFS

2009-11-04T03:19:00.000-08:00

Beware!!! This is not a technical article about how to migrate VSS to TFS.
This is a story about my experience of migrating to TFS.

When we first met with TFS we were using Visual Source Safe 6.0.

Advisors from Microsoft came and introduced the brand new source version control system of Mirosoft, TFS. Future of the source version controls was TFS. Moreover TFS was not just another source version control system. We could even use TFS to manage projects instead of our own project management system. It also had the shelve option when you are leaving work at night. TFS brought everything we needed. They urged us to migrate to TFS because Microsoft itself was migrating to TFS. Because it was newer and better, we should quit VSS and use TFS instead.

One of my colleagues, Yakup, also told us that he used TFS in his previous company. They were pleased with its incremental get ability. For larger projects it was an ache in the stomach to get the latest version of your project in the morning. VSS was checking every file and calculating checksums in order to decide if they were the latest versions. It was taking too much time. TFS was keeping history of your workspace to decide if you have the latest version or not. It was much faster than VSS to get the latest version of a project.

As a punishment to his act of recommending TFS to us, Yakup was chosen to install TFS :-)

After a humble installation (on a poor virtual machine) we started using TFS. Its performance was satisfying. Our project managers investigated project management aspects of TFS but they were not satisfied with them because TFS was not able to keep how much time we spent for each task/bug. They had to report to higher administration of how much a project costed in terms of our salaries :-)

Still we decided to use TFS because of a few reasons. Microsoft advised us to use it, it was new, software companies were going to use it and we did not want to be backwards amongst others, and TFS had the ability of incremental get.

We had two branches, Development and Production. We were developing new projects in Development branch. After finishing a project we were checking out Production branch, moving newly added codes from Development to Production with the help of a folder comparison tool, and finally we were checking in Production branch completely.

Parallel development on the same project still was a big problem. Developers were using each others methods in the Development branch. But some of the methods they use for a feature, say feature X, were happening to be implemented for another feature originally, say feature Y, and because feature Y was still being developed those methods did not exist in the Production branch. Therefore deployment date of feature X was being postponed to the deployment date of feature Y. There were delays on project plans just because we were not able to calculate such usages of other features.

Finally, we realized that TFS was not a brand new source version control system. It solved none of our big problems. We were still having difficulties in parallel development.

In fact we were not aware of the power of TFS, branching and merging ability. The way we used TFS was just like the way we used VSS.

The main purpose of this writing is to make you aware of this single mistake: we did not try to learn conceptual facts about source version control systems. We were just happy with the old VSS. We did not realize the need of some other development methodology for our large scale projects for which parallel development is unavoidable.

In my example, the software project was the whole banking system of a bank. As you may easily guess, all the products of a bank needs software development behind. Competition between banks forces them to present new products to their customers continually in order to keep customers loyal to themselves. Each department of a bank invents new products every once in a while and requests software supply for those new inventions. Therefore huge software projects for banking systems need continuous development to present new products one after one (new account types, new bill payment systems, new investment methods, new credit cards...)

to be continued...
continuing (after a long while)...

Our software development pattern is like this: we have a number of products, and we continually add new features to them. Version release rate is once a month, but bug fix release intervals are shorter.

Thus we adopted such a branching structure:
For every feature request or bug fix, we start a new branch from code repository. Number of branches became, well..., a little too many :-)
In order to manage these branches, in such big numbers, we developed a command line tool using .net APIs of TFS.

Although we have so much branches, we do not encounter merge conflicts that much. This is mostly because a business department makes one or two requests for a certain amount of time (say a year). Plus not all products' codes need to change for one feature. Thus merge conflicts occur occasionally.

Additionally, branching for every request provides us flexibility in releasing features. We can release a feature immediately after development finishes or at a later time. We can even freeze the development if the business department wishes. Newly developed codes are not lost (they stay in the branch) and they do not interfere with other developments (we did not merge the branch to the trunk). If we want to release a feature, we simply merge the branch to the trunk.

We also used the build engine of TFS to check the health of our codes. After every checkin, we start builds and do code analysis, and we also report analysis results to the developers. If a build fails, we alarm all the team, not just the developer who checked in the code. Because the code they are going to get from TFS is going to be dirty, they should not get the latest version until somebody makes sure that the code on the server builds.

In summary, if you are not familiar with source version control concepts, and want to migrate from VSS to any modern source version control tool, not just TFS. I suggest you to search for branching strategies in accordance with your development model.

Get Database Schema

2009-11-03T03:26:00.001-08:00

If you want to list the tables of a database and columns of a table you may use the "GetSchema" method of System.Data.Common.DbConnection class.
It works on both MS Sql Server 200 and 2005. I did not try on other servers.

Sample Code

DbConnection con = new OleDbConnection(connetionString);
DataTable tables =
con.GetSchema("TABLES", new string[] {null, null, null, "BASE TABLE" });
DataTable columns =
con.GetSchema("COLUMNS", new string[] {"CustomerTable", null, null });

Dictionary Performance

2008-01-05T03:29:00.000-08:00

I tested performances of these classes :

A) Key-value keepers
- Dictionary
- SortedDictionary
- SortedList
B) Value keepers
- List
- HashSet
- List (Sorted and binary searched)

I inserted a number of random numbers to these classes and searched for numbers afterwards.
Below results show the average of 100 tests.

Item Count		5		10		50		100		1,000		10,000
Operation		Insert (µs)	Search (µs)	Insert (µs)	Search (µs)	Insert (µs)	Search (µs)	Insert (µs)	Search (µs)	Insert (µs)	Search (µs)	Insert (µs)	Search (µs)
Key-Value Holders(string - string)	Dictionary	3.97	12.81	4.33	11.20	8.75	15.10	12.78	20.04	111.49	116.07	1,037.95	1,082.61
	SortedDictionary	2.27	2.59	6.95	6.30	65.43	48.79	131.48	134.95	2,133.92	1,936.47	29,699.97	27,179.20
	SortedList	2.67	3.58	6.59	6.75	82.51	62.68	116.64	115.66	2,747.47	1,772.13	115,544.27	24,216.48
Value Holders(string)	List	2.46	12.04	2.60	11.89	3.61	42.94	4.62	141.05	33.44	11,644.09	263.98	1,166,795.60
	HashSet	0.84	1.67	1.33	1.70	6.53	5.93	9.76	10.84	78.95	103.39	914.99	1,085.26
	List(Sorted)	9.41	7.37	18.38	11.42	95.63	61.45	184.43	112.95	2,353.52	1,687.84	31,121.19	23,464.25

For value holder classes :
- ICollection.Add(item) method is used to insert items.
- ICollection.Contains(item) method is used to search for items

For key-value holder classes
- IDictionary.Add(key, item) method is used to insert items.
- IDictionary.ContainsKey(key) method is used to search for items.

RESULTS

If you want to keep key-value pairs, you obviously want to do search business.

If you keep less than 10 items of key-value pairs, search performance of sorted dictionary is the best. Above 10 items dictionary beats others in search business.

If you want to keep only values, there are two businesses : transferring the data, or searching the data.

If you are to insert the items to a list, transfer them to some other method, and in theat method you loop through all items, List is the best tool.

If you are to insert items to some list and to do checks of existence on that list, HashSet is the best tool.

Edit : You may find the code here :

using System;
using System.Collections.Generic;
using System.Data;
using System.Data.Common;
using System.Data.SqlClient;
using System.Diagnostics;
using System.Linq;
using System.Runtime.InteropServices;
using System.Text.RegularExpressions;
using System.Threading;
using System.Collections;

namespace ConsoleApp
{

public static class Program2
{

private static void DebugWriteLine(string message, int testCount, VoidMethod method)
{
Stopwatch t2 = new Stopwatch();
t2.Start();
for (int j = 0; j < testCount; j++)
{
method();
}
t2.Stop();
Console.WriteLine(message + "\t" + ((t2.Elapsed.TotalSeconds * 1000000) / testCount).ToString("0.###############"));
}

private static void TestDictionaryInsert<T>(T dict, List<string> items) where T : IDictionary<string, string>
{
foreach (var item in items)
{
dict.Add(item, item);
}
}

private static void TestDictionarySearch<T>(T dict, List<string> items) where T : IDictionary<string, string>
{
foreach (var item in items)
{
dict.ContainsKey(item);
}
}

private static void TestListInsert<T>(T list, List<string> items) where T : ICollection<string>
{
foreach (var item in items)
{
list.Add(item);
}
}

private static void TestListSearch<T>(T list, List<string> items) where T : ICollection<string>
{
foreach (var item in items)
{
list.Contains(item);
}
}

static void Main(string[] args)
{
TestPerformanceOfDataHolders(100);
Console.ReadKey();
}

public static void TestPerformanceOfDataHolders(int itemCount)
{
var items = new List<string>(itemCount);

Random r = new Random((int)DateTime.Now.Ticks);
int random = 0;
for (int i = 0; i < itemCount; i++)
{
do
{
random = r.Next();
} while (items.Contains(random.ToString()));
items.Add(random.ToString());
}

List<IDictionary<string, string>> dictList = new List<IDictionary<string, string>>();
dictList.Add(new Dictionary<string, string>());
dictList.Add(new SortedDictionary<string, string>());
dictList.Add(new SortedList<string, string>());

List<ICollection<string>> colList = new List<ICollection<string>>();
colList.Add(new List<string>());
colList.Add(new HashSet<string>());

var sortedList = new List<string>();

foreach (var item in dictList)
{
TestDictionaryInsert(item, items);
}

foreach (var item in colList)
{
TestListInsert(item, items);
}

sortedList.AddRange(items);
sortedList.Sort();

const int testCount = 100;
foreach (var item in dictList)
{
DebugWriteLine(item.GetType().Name + " Insert", testCount, delegate
{
item.Clear();
TestDictionaryInsert(item, items);
});
}

foreach (var item in colList)
{
DebugWriteLine(item.GetType().Name + " Insert", testCount, delegate
{
item.Clear();
TestListInsert(item, items);
});
}

DebugWriteLine("List(Sorted) Insert", testCount, delegate
{
sortedList.Clear();
foreach (var item in items)
{
sortedList.Add(item);
}
sortedList.Sort();
});

foreach (var item in dictList)
{
DebugWriteLine(item.GetType().Name + " Search", testCount, delegate
{
TestDictionarySearch(item, items);
});
}

foreach (var item in colList)
{
DebugWriteLine(item.GetType().Name + " Search", testCount, delegate
{
TestListSearch(item, items);
});
}

DebugWriteLine("List(Sorted) Search", testCount, delegate
{
foreach (var item in items)
{
sortedList.BinarySearch(item);
}
});
}

}

}