I was very happy when DLR would be the carrier for domain specific language (DSL).  But, why did the general purpose languages like Ruby and Python are added as DLR in the .NET sandwich?  Wouldn’t be like C#, VB.NET or F#?  Not sure, but I was planned to use DLR for one of my custom language implementation using ANTLR.

Today I heared the news that Jimmy, the core developer of IronRuby was resigned from Microsoft.  The one and only one reason is Microsoft does not want to encourage these languages in the .NET world.  Jimmy wrote reasons about this (http://blog.jimmy.schementi.com/2010/08/start-spreading-news-future-of-jimmy.html).

Not sure about Microsoft’s roadmap.  Would it move IronRuby from DLR stack to CLR stack?  What would be the feature and scope of DLR?  Will it be another LINQ to SQL initiative?

But as a developer, which one is best….better….or right platform to learn and develop.  If you think, Android or Apple iOS4…you may be wrong, what about Windows Mobile…not sure until Windows Phone 7 will arrive.   A couple of years before Intel launched Moblin based on Linux but later it was merged with Nokia’s Maemo result MeeGo.  However, if you see the Q1 market share of these, do you know who is still dominating the market….Symbian (44%).  The Apple owns 15% followed by Android with 10%.  However, the trends are in reverse, Apple and Android are rocking.

Oh…then on what basis one can choose the right platform to bring his dream mobile application to the world!  Of course, with quick productivity with rich framework and tools (like Visual Studio & .NET or Ruby on Rails).  What will happened to my rich-web application?  Still, am I kept in touch with ASP.NET, PHP, RoR?

On the other side, Qt4 and J2ME look promising in “one skillset on multiple appliances”.

If you are in .NET world, Windows Phone 7 might be….but have to wait this yera fall.  MonoTouch is bonus for iPhone world. Java….stick with J2ME…sooner Android.  Not sure about iPhone.

Anyhow, be ready to work with at least three skill sets.

Social integrated within the system using Gwibber so that we can enjoy twitter, face book natively.

Booting is boosted with some major tweaks for example HAL has been removed during the booting.

Though this is far from Windows 7, I like it along with Win 7.

Concept behind SCA

Following are the key aspects of SCA:

• Services – a set of operations that perform logically related tasks
• Components – contains the implementation and configuration of services
• Composites – responsible for configuring services
• Domain – A physical environment where the components will be hosted for consumption

A service contains a service contract and address.  Service contract specifies the operations available in the service, the inputs for which to operate and the promised outputs.  Service address provides where the service contract is available for consumption.  Components is nothing but C++ libraries or Java JARs.  Composites are XML file which contains the map between service contract and its implementation.  The domain enables to manage the components and apply policies for their access.  It provides common communication infrastructure and extensibility stuffs.

Technology behind SCA

Although in the specification it has been mentioned that SCA is technology and language independant, it is majorly on Java.  It has been mentioned that SCA is a concept as well as a framework but does not contains anything for presentation layer and data persistence.  However, it integrates with Java based UI and DB framworks.

So…What?

Instead of web services, SCA lets you connect and composite through web services, RPC or anyother means, unlike today protocol based SOA approach.   This is what mentioned and marketed in the SCA specification.  When you are crossing the border of “Concept behind SCA” section of this post, you have surprised that what is new in SCA for SOA, since WCF  incorporated these few years back when this specification was only on the paper.  More interesting and confusing thing is the specification team invited Microsoft for adopting this into .NET.  One more buzz is that this specification is based on WSDL.  These two make me what else have been proposed apart from what WCF has now with such the level of maturity.

For those who want clear map between above concepts and WCF:

• Services - C# interface with WCF declarations
• Components – C# classes with WCF declarations
• Composites – Service model section in app.config/web.config file.
• Domain – Service host (IIS/WAS/Windows Process)

What does it has….What doesn’t?

It provides the following:

• Service reference – dependency with other services
• Composition – way to encapsulate technical/vertical components such as validation, logging and auditing components within service and hide these to the outworld.
• Policy - a statement that controls the operations that provided by the infrastructure.
• Wire – a way for service aggregation and topologies (oneway, duplex, request and response)
• Bindings – provides the means to communicate to the consumers.

It doesn’t have the following:

• Service orchestration – but supports JSR based on BPEL
• Service extension
• Service inventory

Unfortunately, WCF has all the above with the power of .NET platform.  So, beware of SCA jargon in .NET world, where an elephant WCF has all the capabilities.  As of my understanding, SCA is WCF avatar in Java world.

Michael Rowley saluted the WCF proposals and its features, however he tried to oversight it with SCA.  Its confused.
CodeProject

In the above diagram, you can see two regions in Windows. The kernal mode and user mode, and you know that processes in the kernal mode touches the CPU and other hardwares without any interface and have the rights to access physical memory without any virtual address mapping. The user mode processes are our own applications along with Windowing Subsystem which also includes Windows Network Subsystem.  Processes in the user mode requires processes in the kernal mode such as thread scheduling, memory, cache or IO related activities. So, it would require a thread context switching means that kernal model thread has been created and the data in the user mode thread has been transferred into kernal mode thread.  Ahhh, I’m explaining too much about Windows processing.  Lets cut off.

IIS 7 Components

Application Pool and Worker Process

Worker process is a windows process which is specifically designed for hosting and running static, ASP or ASP.NET web applications in IIS.  w3wp.exe is responsible for loading the appropriate ISAPI (a lower level programming module for handling specific web application, for example aspnet_isapi.dll for ASP.NET) filters and starts a new worker process for a request.

Application pool is a grouping of web applications those are routed to one or more worker processes.  Processes in one application pool will not be impacted by another application pool’s failure.

Request Processing Pipeline and Modules

A request processing pipeline is created for every request.  In the previous versions of IIS, if a request is for ASP.NET application, the application needs to exit from the IIS pipeline and load aspnet_isapi for handling ASP.NET code which inturn creates its own request processing pipeline.  In II7, both IIS and ASP.NET request pipelines are combined and called as “Integrated Pipeline“.  This pipeline contains one or more modules.  Module is a component which contain a specific set of features for IIS to handle a request.  For example, BasicAuthenticationModule is a module for performing basic authentication and FileCacheModule is a module for caching files and file handles.

The integrated pipeline has following benefits:

• Eliminating the duplication of features between IIS and ASP.NET, for example authentication
• One inventory for all IIS modules
• Performance…performance…

WWW Service and WAS

It is necessary that a dedicated component is required for listening for requests and managing application pools and worker processes.  In IIS 7, WAS (Windows Process Activation Service) is dedicated for this.  Requests coming from different transports like HTTP, TCP/IP, NetPipe and MSMQ are handled by respective listener adapters configured in the WAS.  When a request is coming which is directed to respective listerner adapters which passes the request to appropriate worker process, where the application manager directs the request to the specific application.

WWW Service (World wide web publishing service, simply w3svc) is the listener adapter for HTTP.  Apart from this, it manages the configuration stuffs required for HTTP related.

Protocol Listeners and HTTP.sys (Kernal Mode)

As its name reflects, protocols listeners receive appropriate requests and passes them to the IIS.  Note that protocol listener are kernal mode processes.  This model allows to handle any protocol, and the only thing we require is appropriate device drive as like as HTTP.sys for HTTP requests.

Whenever a HTTP request is entering into the server, HTTP.sys receives the requests and pass it to appropriate worker process if exists, otherwise it puts the request in the request queue.  When a response for a request is already in the response cache, it fetches that and reply it back without noticing and invoking worker process.

What is AppFabric?

A set of components provide service hosting, in-memory caching, stateful services persistence and management services, management consoles and API to access these services.  The following figure shows the major responsibilities of AppFabric components.

Hosting services extends WAS (Windows Process Activation Service) with some default configuration, workflow management and, administration tools.  In-memory caching, previously code-named as Velocity, is a solution for scalable caching of enterprise/web applications.  Its cluster architecture and provision to configure various caching options enable us to prepare feast for small to large crowd.  Not know in detail about what makes different from WF 3.x and SQL Server combo, and state persistence service in AppFabric.  Let us see in detail on future logs.  To manage these aspects, AppFabric provides variety of management services which includes dashboard, service deployment, monitors and APIs.

Setup your AppFabric Environment

Windows Server AppFabric is actually designed for Windows 2008, but has development stage avatars for Windows Vista/7.  AppFabric beta 1 targetedc .NET 4.0 beta 2 and AppFabric beta 2 is targeted .NET 4.0 RC.  It will act as extension of IIS 7.0.  The following figure shows the IIS MMC for this.

The Windows 7/2008 beta 1 can be downloaded at http://download.microsoft.com/download/0/A/E/0AEB3BC1-506E-4954-8AB1-4FA2EE75985C/AseSetup_x86_6.1.exe.

Beta 2 can be downloaded at http://download.microsoft.com/download/F/D/4/FD4F05A5-016B-40F8-A61F-1D38E202A32A/WindowsServerAppFabricSetup_x86_6.1.exe.

static void Power(int v, int p = 2)
{
 Console.WriteLine("{0} Power of {1} = {2}", v, p, Math.Pow(v, p));
}

p is the optional parameter here. By default Power does calculate square of v, if p value provided explicitly, does calculate v to the power of p.

Power(2, 3);  // 2 ** 3 = 8
Power(4); // 4 ** 2 = 16

Actually, two parameter target attributes help to make this feature available. These are:

• OptionalAttribute
• DefaultParameterValueAttribute

in System.Runtime.InteropServices namespace.

The OptionalAttribute marks the parameter as optional as like

static void Power(int v, [Optional] p) {...} // p's default value is to default(int) i.e. 0

Power(5); // prints 1

The DefaultParameterValueAttribute constructor takes value parameter of type object is used to initialize the default value of the target parameter.

static void Power(int v, [Optional][DefaultParameterValue(2)] p) {...} // p's default value is 2

Points to be noted on “optional” parameter usage:

• Optional parameter should be the last parameter as like “param” usage

ref and out parameters.

Named Parameters

Optional parameter will not be useful in real without named parameters. Yes, the name itself tells the answer. What will you do if you have more than one optional parameters and at the calling place, you want to pass actual value for less number of optional parameters. For example

static void MoreOptionalParamMethod(int a = 0, int b = 1, int c = 2)
{
 Console.WriteLine("a= {0}, b= {1}, c= {2}, ", a, b, c);
}

Now, you want to pass the value of variable b only. Named parameter will help you.

Instead of parameter order, you can explictly specify parameter name along with the value using “:” in the calling place

See the following calling place

MoreOptionalParamMethod(b: 5); // prints a= 0, b= 5, c= 2

There is no surprising in the IL code, the compiler replaces optional parameters with default values as like below

MoreOptionalParamMethod(0, 5, 2);

The only thing to be considered is named and normal parameters can be mixed but normal parameter should be come first.

CodeProject

Recently, I was in a situation to find out suitable caching solution for our WCF based service layer.  Although, Microsoft yet to avail its “Windows Server AppFabric” to the .NET/WCF world (which might be the filler of .NET peoples’ decade old expectation), there are some caching solutions available from third parties. 

I did a small evaluation on the following caching solutions:

• NCache Express
• Shared Cache
• Memcached
• Enterprise Library 4.1

Disclaimer: The test was done at very small scale and not included the scalability factors.  Memory and CPU consumption were not covered.

NCache supports different caching topologies and types with many features like Object Query Language, remote clustering, etc.  I’ve taken its express edition for this evaulation.  Shared Cache is 100% written in C# with distributed caching and three different topologies.  It supports WCF DataContractSerializer in addition with BinarySerializer.  Guys from *nux world knows the defacto caching solution “memcached“.  It has Win32/64 avatar too and some open source managed client libraries also available.  I used Danga Interactive 32-bit version of memcached 1.2.6 (http://code.jellycan.com/memcached/) and BeIT Memcached managed client (http://code.google.com/p/beitmemcached/).  64-bit Windows version is available at http://labs.northscale.com/memcached-packages/.

I’d done a small performance testing on the above solutions.  In addition to these, I’ve used Enterprise Library 4.1 CAB provider model and implement provider for these solutions.  So, following different solutions had been tested:

• Enterprise Library 4.1 CAB
• Memcached
• Enterprise Library 4.1 based Memcached
• Shared Cache
• Enterprise Library 4.1 based Shared Cache
• NCache Express
• Enterprise Library 4.1 based NCache Express

Following three different test had been done on every solutions, totally 21 test:

• Adding 1000 items & Getting these items
• Adding 1000 items & Getting 5th element
• Adding 1000 items & Getting 995th element

The testing was performed on Intel Pentium 4 3.2GHz, 2GB RAM running WinXP SP2.

Let us see the results and, pros and cons of these solutions.

Enterprise Library 4.1

Not surprisingly excellent performance due to in-process memory caching, however not suitable for enterprise scale.  This supports primitive and Serializable objects.

The “R” deonotes “Run”.  I did 4 runs for every test.

Memcached

Since, the OS calls for *nux memcached and Windows memcached is different, some bloggers cautioned memcached on windows is might not be a win-win.  But, I could not find any degrades in my testing.  It takes small memory foot-print and CPU usage.  It supports primitive and Serializable objects.  However, the Win32 service version does not have flexible configuration.

Shared Cache

Like Enterprise Library 4.1 CAB, this solution uses provider like model and come out with IndexusDistributionCache which supports WCF data contract serializer too in addition with binary serializer.

NCache

This is more matured and tailored solution for .NET world.  The best is the object is not necessarily Serializable.

The results shows Memcached performed better for its consistent performance and simplicity.  The surprising fact is that the above all solutions perform well in Enterprise Library 4.1 CAB provider model than their default mode.

Its released for Linux and Mac only.  After downloading the source from https://go.googlecode.com/hg, building the packages and set some environment variables (guide @ http://golang.org/doc/install.html), tried a small sample which greets the arg(0) given from console.

The syntax is neither smart nor friendlier at my first sight.  Seems it is a mix of JavaScript, Java and ofcourse, C++.  For each and every compilation you need to specify target machine architecture, 6g – for AMD, 8g – for 386.

See the following sample hello, <user> sample:

package main

import (
"fmt"; // formatted stdin
"flag"; // command line parser
)

func main(){
 flag.Parse();

 fmt.Printf("hello," + flag.Arg(0) + "\n");
}

After the compilation with 8g, invoke the linker 8l and generate the executable (normally 8.out).  Since I did not evaluate this language at some level and did not read its architecture, I could not give any comments of this early stage compiler. However, at my first sight,

GO = Neither C# Nor Ruby

PLINQ

PLINQ has implementation of  all LINQ to Objects extension methods in System.Linq.ParallelEnumerable for IEnumerable.  To make an IEnumerable parallel, you simply call AsParallel() extension method.  AsParallel() internally calls System.Linq.Parallel.ParallelEnumerableWrapper which is of type System.Linq.ParallelQuery<TSource>.  Let us take the same customer-order example as shown in my previous post with  PLINQ enabled.

var customersForOrderId7 =
    from c in customers.AsParallel()
    from o in c.Orders
    where o.OrderId == 7
    select c;
 

customersForOrderId7.ForAll(c =>
    Console.WriteLine(c.ToString()));

The above code selects customers who have ordered OrderId 7.  AsParallel() at line 3 makes Customer[] as parallel enabled.  In this example, you can see that instead of typical for…each, I’ve used ForAll().  This differs from for..each, means that for..each preservs the final order of the results.

Thread Pool and Work Stealing Queue

Until .NET3.5, we can queue our work items as thread in to ThreadPool using QueueUserWorkItem().  In .NET 4.0, improvements had been made in the thread pool.  It is now based on System.Threading.Tasks.Task. 

Task

Task is a work item which can be executed independently along with other tasks, technically it represents an asynchronous operation i.e. System.Action.  It is defined in System.Threading.Tasks.Task which enables you to do the following actions on a task instance:

• start – to start the instance
• wait – to wait to for a task to complete
• cancel – cancel the task asynchronously

It seems that task is a lite version of thread.

Thread Pool

Following diagram shows the conceptual view of .NET 4.0 thread pool.

Previously, thread pool had only one queue on which all the work items were queued and enqueued in FIFO order (ofcourse, thats why queue).  The worker threads are allocated for every work item access the work item from this queue.  In .NET 4.0, it has been improved by introducing local queue for every worker thread, in addition to qlobal queue.  Tasks those are created by program thread queued on global queue.  The task scheduler enqueues the tasks from global queue in FIFO order and distributes to respective worker thread’s local queue.  The worker thread enqueues the tasks from its local queue in LIFO order.  Interesting!

The introduction of local queue makes these threads can be executed on different processors without contention issue which normally occur in single queue thread pool.  The reason for worker thread picking up the tasks in LIFO order is the assumption that “last-in” is hot to act which results no qurantee in task ordering, but better performance.

Work Stealing Queue

Let us assume that worker thread 1 completed all of its tasks in the queue.  It scans the global queue for any task and if nothing there, it scans other worker thread’s local queue.  In this case, let us assume there are two tasks in WT2 queue.  WT1 enqueues first-in task from WT2 queue which avoids contention issue.  Getting tasks from other local queue is called as work stealing.  This again results better performance.

References

MSDN article: Task Parallel Library Overview: http://msdn.microsoft.com/en-us/library/dd460717(VS.100).aspx

Daniel Moth’s videocast and PPT deck: http://channel9.msdn.com/pdc2008/TL26/ and his article about thread pool at http://www.danielmoth.com/Blog/2008/11/new-and-improved-clr-4-thread-pool.html

My Code sample for PLINQ as used in this post is at http://www.udooz.net/file-drive/doc_details/9-tpl-and-plinq-example.html