ReBuildAll
LenardG's thoughts (mostly) on .NET development

This article will discuss my opinion on Nokia and Windows Phone. If someone reading it notices that I got my facts wrong, please do correct me

Updates to this article appended to the end.

A Hungarian proverb

There is a Hungarian proverb that translates roughly like so: one of my eyes if crying, the other is laughing. It refers to some kind of thing which is both good and bad for a person. And that is what I feel when I think about Nokia and Windows Phone adoption.

Crying? I am just not reassured, that Windows Phone will be so great for Nokia. This feeling might be caused by my previous clash with the Windows Mobile platform (pre-Windows Phone 7), that you can read about here. This obviously does not apply to Windows Phone 7 (or at least not entirely), but I can't help my bad feelings. There are some features of Symbian (we will get into this later) that are missing from WP7 (and also from iPhone or Android).

I will admit I never used a WP7 phone for more than a short time in a shop or lending it from a friend to try things out. I have also done some research (reading about it, reading phones manuals). My opinion is based on these facts. But I still have that same feeling which I had with Windows Mobile previously: that you are not in control. The phone (or Microsoft) does certain decisions on how you should use your mobile phone, and the restraints just make me more angry I know many would recommend Android to me based on these facts. Maybe. (I will also admit at this point, that some features I critisize here might already be available on Android, or might be available via separate apps, etc. My Android information is dated )

Laughing? So why would I be happy about this move of Nokia? I have been developing in .NET since 2004, both professionally and for a hobby. I really really like the platform, and Microsoft's strategy to have the same programming APIs across a wide range of devices is just really tempting. I do not need new tools or new skills for that matter (as I have programmed Silverlight for example), I can just hop in and start coding Windows Phone 7 (I have written some apps in the emulator already). And I can soon run it on a Nokia, which sounds pretty interesting.


Profiles and groups - are features dying?

Wow, Windows Phone 7.5 Mango now supports grouping your users. That is awesome. Nokia phones did it 10 years ago, and in my opinion better too. In Mango groups serve only to be able to view a group of users together (their socials updates, etc) and to communicate with them. Granted, groups might not be integrated into the social aspects of newer Nokia phones, but what about basic "PHONE" settings? Like setting a ringtone for a group? Like selecting which groups will alert you (ring) in a given profile?

... Oh wait ...

Windows Phone does not have profiles? Well it sure seems so. Set a ringtone. Check. You can have it on silent. Check. You can have it on vibrate. Check. You can't use vibrate and silent at the same time. What?

If I want to have silent available, I have to TURN OFF Vibrate in settings? To have vibrate on again, need to set settings again. Otherwise, the volume down button will put you into either SILENT or VIBRATE. Frankly, that seems just stupid. And I wonder if anyone who wrote this part of Windows Phone has ever seen how Nokia does profiles? (yes, I doubt the person has seen that feature in action).

My only hope is, that Nokia will port these missing features from Symbian into their Windows Phone offering. That would give them a huge edge over other Windows Phone manufacturers too - at least for some time, as they have to contribute those changes back into WP7. But that would just make WP7 better. However, I am afraid this porting over features will not happen - but who knows?

And do not even get me started on things like "who needs profiles?" and "people do not use their phones to make phone calls anymore", because those things are just stupid.

To sum up this section, I think that Symbian offered a lot of control over your settings and a lot of settings to choose from and customize, how your phone works. Loosing these things in the transition to Windows Phone would be unacceptable to me, but we will have to see what the good guys in Nokia think about this. Granted, the settings were not logically organized and often hard to find, but they were still there. I dislike the "we know better than the user" design philosophy, which is visible in both Windows and Windows Phone.

If features from Symbian end up missing from the WP7 Nokias, that is just a downgrade, plain and simple.


Why Software Sucks?

But why does Nokias software suck?

As a side note, there is an excellent book called Why Software Sucks? I do recommend anyone involved in software development to read it

But back to Nokia. Why doesn't Symbian work so well? I would think this is because the systems core is ancient (at least by computing standards) and building on a not so solid foundation is just not that easy. But Nokia has some capable people, and you can see the results by looking at the N9 and MeeGo. You have to wonder where Nokia kept that software talent all these years, because that thing really works well. Also note, that my criticism is only for the software side of Nokia phones. Their hardware is really something to admire, especially recently. I just wished their software would been on par with the hardware they make.

Nokia handled some things concerning their software badly. The software development speed, update cycles and software philosophy left a lot to be desired. I have seen it many times, that old phones just do not get all the nice updates new phones have gotten. This is now changing. In the Elop world, Symbian^3 devices get all the updates that newer phones come out with. Sweet. I hated that you need a new phone every 6 months to get the new features. It seems now all manufacturers are updating - more or less - old models with new software. It might seem like loosing money, but it is building up customer satisfaction. Besides, people often get locked into contracts, and cannot upgrade their phone so often. By releasing new software for new phones only, you just make those people angry.

The software updates with Nokia are coming SO SLOOOOW. There are bugs to fix, applications to update, and the updates are just coming very slow. When Symbian Anna finally arrived in the summer, it arrived one operator, one phonemodel at a time, and many months later than promised. I have been reading Nokia forums and found out, that they need to test every variant of the software. Presumably by hand. And there are variants for different color versions of the phone. Yes. You read that correct.

I see Nokia wasting money here. Money and time. And by that, customer satisfaction.

Building packages for all phone models, variants and operators SHOULD BE automatic. These variants then SHOULD BE tested automatically. I would have expected that a company like Nokia can press a button at night, and in the morning see if everything went ok, without a single person being involved in the software testing process. All packages should be built and packaged automatically and then tested, also automatically. Judging from the speed they do things, it is just not how things progress. (The reality of testing mobile phone software might be more complex than what I described above, but still it should be more easier than it seems to be now.)

Because the update process is such a big pain, of course they will not want to do it too often, so what we get is updates coming really slowly.


A new software platform again?

It was in the news that Nokia is making a cheap, simple, Linux based platform that it wants to put on the cheaper phones. And you know, this has me wondering. Why oh why do we need ANOTHER PLATFORM? You already have MeeGo, don't you? Instead of thinking about adopting that for cheaper phones, you go ahead and start developing something new again? Excuse my language, but that does seem like bloody stupid to me.

The whole technology industry is about product lines.

Take hardware. Intel processors. If in the production some processors do not qualify high enough, they have some features disabled and labeled cheaper/lower end models. Intel is even selling processors where you can unlock faster speeds AFTER you bought it by paying for an unlock code. I do think that practice is disgusting, but it does point out one key thing. They do not design and manufacture as many processor types are they sell. They simply lock out features of high end models to make them low end. Same goes for graphics cores.

Take software. You create a nice little program with a lot of feautes. Name it the Enterprise version. You lock some features down, that is your Professional version. You take away more features, Home version. And heck, make even a free Express version too. You can see it all over the place. Microsoft does it with Windows, and most independent software vendors do it too. It is much easier to lock certain things down than to create entire new versions. No one really thinks Windows Home Premium is a complete rewrite of Windows Ultimate, with certain things left out?

Why does not a company like Nokia go this way? Why make something new, when looking at the N9, you already have a very mature and good operating system. Use that. Lock things down, but use that.

The same argument goes for the now dying S40 platform. I never quite understood why you need a different platform.

There might be certain hardware restrictions to work with - like cheaper models not getting so strong hardware - but I will argue that having a single codebase is still cheaper than multiple code bases. Cheaper to make, cheaper to maintain and test.


A mistake?

I will go as far as to saying Nokia's decision to adopt Windows Phone seems like a mistake to me. Looking what Nokia has available today, it is very clear that these things have been in the making when they decided to go the Windows Phone 7 way. These offerings available now seem to me that these might have been the solution to their problem, and not adopting Windows Phone 7. Frankly, it makes you wonder about all the theories concerning why Elop chose his former employer's mobile platform.

I will explain.

Nokia now has MeeGo ready. N9 has appeared in stores. If there would be no Windows Phone 7 in the making, MeeGo devices could now appear one after the other. Read the reviews of the N9 over the internet. Praises. The user interface is compared to the iPhone, the user experience is said to be great. And some people are sad because it is not available in their country. The reaction would have been the same without Windows Phone 7. In fact, it would be even much much better, because everyone could look forward to improvements to the MeeGo platform and new devices. Not today.

On the development side, S^3 and MeeGo can be both targeted using the same development tools (Nokia Qt). So developers really do not need a lot of effort to transition. The development tools BTW also came a HUUUGE way forward, they are easy to install, maintain and use. Ovi Store is available today with lots of apps.

So right there, you already have a complete mobile platform and ecosystem available. But Nokia had to destroy it all. I do not believe for a second that four mobile ecosystems would have been too much.

Of course MeeGo cannot work in its current form, because both platforms (S^3 and MeeGo) are essentially dead ends. Current developers will work with them, but I really doubt anyone new will invest lots of time and money in training new developers for these platforms.

So, instead of the already ready and fine working MeeGo, we get Windows Phone 7 Nokia phones, that will maybe work OK, but will certainly need time to fine tune and get used to. I cannot even imagine when Nokia will be able to make a phone like the N8, with all those multimedia features: photo picture quality, video editing features, etc. As far as I know, WP7 does not have video editing?

Frankly I did not see the driving reason behind WP7 adoption back in the beginning of the year, and I see it even less now. I think Nokia threw away a tremendous opportunity by choosing Windows Phone 7, and I really doubt they can climb up to top spot again. Very sad actually.

I can really understand the comments by former Nokia chief Anssi Vanjoki who also said Windows Phone 7 was not needed by Nokia. If you understand Finnish, read his comments here.

Update 22.10.2011: Engadget just reviewed the N9, coming to similar conclusions: it is just a shame Nokia killed of MeeGo.

I gave Windows 8 a try the other day. While I have not clocked a lot of usage time with it, here are my first impressions.

Login

I liked that I can use my Live id to login, and that it will synchronize my user settings around computers. I like it expecially since at home I have multiple computers, but no domain. With an Active Directory domain you can accomplish about the same (roaming profiles), but until now it was not possible with normals accounts. At least without some custom applications. At least Chrome synchs browser settings (even themes and extensions), so in our web centric world today, that makes it really easier to use many computers. And with W8, this might be even better

I noticed that it does create a local user account even when using Live. It somehow links this local account to your Live ID. I would love to know more about how this integration technically works. Background information anyone?

Start Screen

There is a new Start Screen in Windows 8. There are actually several articles from Microsoft that address the Start Screen, one of the latest being:

Designing the Start Screen

The article explains that study shows, people are moving away from the start menu. People are pinning programs directly to the task bar and launching them from there. This is very true, and I can say that I have been moving in this direction myself. The most frequently used programs are on the task bar. The less frequently used programs are pinned to my Start Menu. And the rest is possibly listed in the recently used portion of my Start menu. And if I still do not find something, I just use Search (press Windows button on keyboard and start typing). It is very very rare for me to venture into the Start Menus. So probably this research does tell the truth and people are indeed using the Start Menu less often.

Ok, so Microsoft now replaces the Start Menu with the Start Screen, which is a sort of dashboard, will all applications on it, live tiles with information, etc.

Continuing the previous use case of rarely used programs: I do have to point out however, that on those rare occasions, it is very easy to find the program I am looking for in the Start Menu. Because if I need something Visual Studio related, I look in that folder of Start Menu. If I need something Office related, I look in Microsoft Office folder. The new Start Screen just lists everything it finds in the Start Menu. So now with this new Screen on the rare occasion I need something thats name I do not remember (cannot search), it actually makes it HARDER to find it, because everything is in one big messy pile of icons. Yes, I can customize the Start Screen, but we are talking about rarely used stuff, why would I want to customize that, when it is already in neat folders in my Start Menu?

It covers the whole screen

But the new Start Screen in Windows 8 has several more disadvantages. As others have pointed out, it COVERS THE WHOLE SCREEN. So I do not see at a glance, what I have already running, and what is not running on the computer. As soon as I launch into the Start Screen, the task bar is hidden. This full screen operation mode does not suit desktops in my opinion. It might suit tablets, but definitely not the desktop.

The article also mentions, that you can still use Win+1 or Win+2 to launch programs from the task bar. That is great, except wait, I cannot see my task bar from the Start Screen. And when I cannot see it, I also cannot use those shortcuts! (well until I visit the desktop at least once, which is not trivial task either)

To make matters worse, not even the mouse works until you CLICK on the Start Screen, which is already visible (BUG?). You cannot close the Start Screen and see your desktop until you start something. After that, even if you exit that, you can use ESCAPE to go back to Desktop or press the Windows button. (again, BUG?)

Notifications

Next, Microsoft says that the new Start Screen brings together notifications, which were poorly implemented in the taskbar popups and notification tray icons in previous versions of Windows. The start screen connects you to apps, it shows news, RSS feeds, weather, and so on.

Now this is very nice, but guess what, when I am using my computer, I do not spend time on the Start Screen. I spend time in Visual Studio, in Word, in Photoshop, in my browser or in Outlook (or another application). There might be more application windows visible on my desktop side by side, maybe even on multiple monitors. Those notification that popup on the screen or from the task bar / notification tray are a way to get my attention. I am not going to check back to the Start Screen to see if anything new has happened. I am not going to sit around the Start Screen waiting for emails or facebook messages. I will be spending time in Visual Studio or in Word or in whatever application I am using, and when that notification icon appears in the task bar icon of Outlook I am going to switch over there and check the mail. And I am not going to go through a Start Screen.

A Start Screen is a good point to START things, but during work or operation, it is not something I want to use, at least not in its current form. Now when it pops up fullscreen, it just obscures everything and gets in the way of getting things done.

While I criticize how it works now, I do see the need and use of a centralized hub of information, where you can check out notifications all at once, check if new news came in, check if there are new bugs, check if there are new messages, chat messages, emails, status updates on Facebook, etc. But I would want that to be less disruptive than launching into the Start Screen. Besides, if I need to switch over to the Start Screen to check for notifications, it just beats the whole purpose of notifications in the first place.

The keyboard experience, searching

It is not all bad - but I will say it definitely needs fine tuning for the desktop use. And it needs tuning to be used with the keyboard.

I am a big fan of using Windows with the keyboard, and I was very satisfied by how Windows 7 improved the keyboard only experience of using the operating system. With the Windows 7 Start Menu, I can push the Start button on the keyboard, start typing (into search), use arrows to select from list, and launch. Often the first result was good enough, so I could just hit enter to start the application. The new Start Screen works like this if what you want is an application. It also displays more results at once. But in certain situations the experience is worse than in Windows 7. Not to mention that the Start Screen covers all apps and the taskbar. Not good.

What I particularly did not like about search in Windows 8, is that it separates Apps, Settings and Files. It just displays results from Apps as a default (in Windows 7, all results were displayed at once). What does this mean? On Windows 7, when I type in “add remove” into Windows 7 search box, the first result is “Add or remove programs”. I can hit enter and start working. In Windows 8 the result is bogus ("Add or remove help content", obviously some kind of App), until I remember to switch over to Settings under the search box. Which is not even trivial using the keyboard.

As mentioned before, Start Screen covers the screen, so naturally so do search results. As the article says, the start menu could not scale to fit all the search results: this is true. In this respect the new Start Screen is superior, because in can display more entries.

I would stress that the task bar needs to be visible all the time. When Windows starts, by all means, show the Start Screen. Maybe even full screen. But almost always I would see a better option of the Start Screen being a transparent overlay (check out OS X Dashboard) over the desktop, excluding the taskbar (which should be always visible).

Summary

This is of course just a developer preview, and I hope Microsoft can fine tune the new Start Screen, especially for desktop users. I would say the keyboard (and mouse) user experience needs a lot of work, because in its current form it is more disruptive than helpful.

When Windows Vista came out, and after that, Windows 7, the new Start Menu and its features struck me as something that increased my productivity, especially with the keyboard. I just do not feel the new Start Screen does the same. I feel it hinders me. I realize sometimes you need drastic changes to get new things going, but I just do no feel the Start Screen in its current form satisfies this goal. It might be something really cool for tablets, but desktop users are left behind in its current version.

Eagerly waiting more Windows 8 previews And I still need to check out development tools for Windows 8.

Having played with BizTalk some while ago (version 2006 R2) I had an interesting problem: in certain situations BizTalk would not accept web service description files (WSDL) where the XML schema was stored in separate files (XSD). No matter how hard I tried, there was no success. I even tried the good old trick of putting the files in a webserver and trying to add them over HTTP. It would not work. (This is the way you trick the Visual Studio proxy generator when WSDL and XSD files are spread over the hard drive in different directories, but that is another story )

Finally, I ended up creating a tool I called WSDLMerge. This can take a WSDL file, local or remote, and merge it with all the XSD files referenced. The merging is recursive, so any XSD files referenced by other XSD files are also included. It can follow local path locations and remote path locations. The result is a single WSDL file, that contains everything.

If you want to jump right into the code part, you can find the tool in source code format at GoogleCode. You will need Visual Studio 2010 to compile, but you can safely run it with .NET 3.5 SP1 (maybe even earlier).

The rest of this post will talk about how this tools works.

Just XML

WSDL files are just XML files after all. So we can go ahead and load it from disk or from a URL.

XmlDocument wsdl = new XmlDocument ();
wsdl.Load ( filename );

And voila, we have the entire WSDL loaded up. We need to create a XML namespace manager because we are going to work with namespaces. XPath searches in particular required the namespace manager. If you have the source code by now (see link above) you can find the following code in a method named PrepareNamespaceManager().

            XmlNamespaceManager manager = new XmlNamespaceManager ( wsdl.NameTable );
            manager.AddNamespace ( "wsdl", WSDLNamespace );
            manager.AddNamespace ( "xsd", XSDNamespace );
            return manager;

The tools will verify if the file loaded is an actual WSDL file. It does this by checking for the root element, which should be wsdl:definitions. There are probably better ways to do this, but this is good enough for our purposes.

Schemas, where are thee?

Next step, find schemas. These can be found under the following XPath /wsdl:definitions/wsdl:types. We read the import definitions one by one, load the schema location and namespace parts of the imports. We also keep track of all namespaces we have already loaded.

Ok, so first we locate the element where we should find the schema import statements:

XmlNode node = wsdl.SelectSingleNode ( "/wsdl:definitions/wsdl:types", manager );

If such an element exists, we can start finding any schemas:

XmlElement schemaElement = typesElement.SelectSingleNode ( "xsd:schema", manager ) as XmlElement;

Here the process turn recursive. This is done using a method called ProcessSchema() that is designed to process a single schema definition.

Inside this method we need to know if the schema is inline or if the schema is imported. So we look for import elements:

            imports = rootElement.SelectNodes ( "xsd:import", manager );

If we find anything, we check the namespace for this schema as well as the schema location. If the namespace we find is not yet loaded, we load the .XSD file (from either disk or an URL), attach it to the main document, and remove the import statement.

XmlDocument schemaDocument = new XmlDocument ();
schemaDocument.Load ( importLocation );

XmlElement newSchema = wsdl.ImportNode ( schemaDocument.DocumentElement, true ) as XmlElement;

XmlNodeList newImports = newSchema.SelectNodes ( "/xsd:import", manager );
foreach ( XmlNode importNode in newImports )
{
                    if ( level == 0 )
                    {
                        newSchema.RemoveChild ( importNode );
                    }
                    else
                    {
                        if ( importNode.Attributes["schemaLocation"] != null )
                        {
                            importNode.Attributes.RemoveNamedItem ( "schemaLocation" );
                        }
                    }
}
schemas.Add ( importNamespace, newSchema );

The ImportNode() method handles duplicating the element from the schema into our WSDL document. We also remove any import elements from the duplicated element (this could mean removing schemaLocation attributes). We do not want any XSD to import anything.

Of course we also do not want to have any schemas missing. So while we remove schema references from the document we are creating, we will want to follow them in the original documents. After this processing done we will process the original XSD (from which we created the duplicate) for these import statements, and call ourself (ProcessSchema()) recursively to import any further XML namespaces.

When this process is complete, all namespaces (== XSD files) that are referenced in any of the directly referenced schemas or anywhere in there recursively will be included one by one in the body of the WSDL document. This sort of flattens the entire XSD structure (previously using files it was built as a tree like structure). The schema references will still be in place, and because all schemas are now in the body, the WSDL will not have any dependencies.

In the end, the whole process is just navigating and modifying XML documents, looking up references, loading them, and attaching duplicated elements and nodes into the master document. This master document will become our merged WSDL document, which we just write to disk in the end.

This entry will be available in Finnish only - it is related to the TechDays 2011 speech on 31.3 about extending MVC framework.

Pidin esityksen TechDays 2011 MVC kehikon laajennuksesta. Esityksessä käsiteltiin miten MVC kehikkoa voi laajentaa eri paikoilla.

Itse kehikko on iso kokoelma pieniä osia mitkä toimivat hyvin yhdessä. Kuitenkin näitä pieniä osia voidaan muokata ja räätälöidä, tai jopa tehdä omia osia mitkä käytetään alkuperäisten sijasta. Näin voimme räätälöidä itse kehikonkin että se toimii juuri niin kuin me haluamme.

Näitä laajennuspisteitä on monta, ja esitys yrittää antaa kartan näihin pisteisiin. Ohjeiden perusteella pitäisi olisi helppo löytää juuri se laajennuskohta, mikä antaa mahdollisimman paljon hyötyä.

Esityksen PDF kalvot ovat ladattavissa täältä.

Esityksen demo sovellus on myös ladattava täältä.

Demo vaatii Visual Studio 2010 version (Express pitäisi toimia myös) sekä SQL Server Express version. Kun sovellusta käynnistää ensimmäinen kertaa, se luo itselleen tietokannan (App_Data kansioon). Kuitenkin pitää käsin ajaa alustus. Se löytyy /Home/Setup osoitteesta sovelluksen alta (eli HomeController:in Setup action).

In response to the comments I have received to the original article, I have updated it with information how to modify the behavior of the jQuery validator plugin without directly editing the source.

While working with MVC 3, I hit some problems with validation. MVC 3 uses jQuery validation plugin with unobtrusive validation to help you out. Except when I had to use Finnish formatted numbers, there was a problem. jQuery/javascript did not really like the comma as a decimal separator.

A similar problem is with the range validation, which also refuses to work when using a comma as the decimal separator.

A rather nasty set of problems I might say, especially since number validation rules are added AUTOMATICALLY by ASP.NET MVC if you render a TextBox for a Decimal field in the model. It just seems the writer of the validator plugin forgot about everyone outside US (thanks @Deef for this last comment).

Fixing jquery.validate.js

You can of course go into the source of the validator javascript file and make your changes there, but modifying this file should never be done directly. That would create all kinds of nasty problems when updating to future versions.

Instead, one should take advantage of extending (replacing) the validator functions that are provided by jQuery Validate. (thanks to @ruben for the information on this).

To fix these problems, we can take the default implementation from the jquery.validate.js file for the range() and number() functions. We then create another .js file (say jQueryFixes.js), in which we override these default functions with ones that contain support for the comma as a decimal separator. The contents of the file should be something like this:

$.validator.methods.range = function (value, element, param) {
    var globalizedValue = value.replace(",", ".");
    return this.optional(element) || (globalizedValue >= param[0] && globalizedValue <= param[1]);
}

$.validator.methods.number = function (value, element) {
    return this.optional(element) || /^-?(?:\d+|\d{1,3}(?:[\s\.,]\d{3})+)(?:[\.,]\d+)?$/.test(value);
}

This should be the last .js file you include, so it overrides the behavior of the default functions from the jQuery Validator plugin.

For number validation, what we have done is replace the last dot with an expression that also accepts a comma (change is bolded):

return this.optional(element) || /^-?(?:\d+|\d{1,3}(?:,\d{3})+)(?:[\.,]\d+)?$/.test(value);

This will solve the number validation problem with the decimal point/comma. If you are dealing with thousand separator, you might need to tweak this further.

As for the range validation, I opted to simply replace the comma with a dot, and proceed normally from there on. You could of course use the jquery globalization plugin, please see the comments section for @ruben's recommendation on that one.

And that is about it. Now decimal separator commas should also be accepted by the validators.

Some time ago I wrote an article about MVC and virtual views. I made a comment that you could create a VirtualPathProvider that would serve content from a ZIP file. Some of you commented you would like to see an implementation for that. Well, here it is.

Please read that article for more information on how to implement a VirtualPathProvider. Here I will not go into details on everything involved.

You can download the entire example solution (Visual Studio 2010 and MVC 3 required!) from here.

The example should run as soon as you load it up in VS2010. Try to request the following addresses (relative to the application root): /Test and Test/Complex for non zipped versions of views. And /TestZipped and /TestZipped/Complex for zipped versions of the views.


How to extract files from a .ZIP file?

For this example, I will use SharpZipLib, which a freely available .zip file management library. It is already included in the example solution I linked at the beginning of this article. So if you downloaded that you will not need to download SharpZipLib separately.


Preparing to serve Views from a .ZIP file

I decided that the content I want to serve will be stored in a file called Views.zip, that needs to be stored at the root of the application (~/Views.zip). This .zip file has to have the same directory structure as the real MVC application. So we need a Views folder, and inside that, the controller folder. Inside the controller folder (or folders) there will be the actual views.

For the example solution I put the views for the "TestZipped" controller into the Views.zip file. There are two views, Index.cshtml and Complex.cshtml. Both include Razor markup to demonstrate that the solution demonstrated is actually fully functional. You could of course pack up the entire Views folder in the example solution, and it will still work

(I did try this, and it did work. If you do it, make sure you still leave web.config in place, as that is outside the scope of VirtualPathProvider. But you can just put everything else in the Views/ folder into the .ZIP file, and delete them from the actual folder. The application will still work! )

The directory structure inside the .zip will make it easy to find and extract the files. Because the VirtualPathManager receives virtual file names, like /Views/TestZipped/Complex.cshtml, when we have the same structure in the .zip, it is easy to string compare the names.

Of course you could implement as complex a logic as you want to find the proper file


Creating a VirtualPathProvider for serving content from .ZIP files

The next step is to create a new VirtualPathProvider. Please note that the implementation I provide is for demonstration purposes only, and might not be fully ready for production use.

    public class ZippedVirtualPathProvider : VirtualPathProvider
    {
        public static void RegisterMe()
        {
            HostingEnvironment.RegisterVirtualPathProvider(new ZippedVirtualPathProvider());
        }

I also created a little helper method to register our path provider.

We will override FileExists(), GetFile() and GetCacheDependency(). Let's start with FileExists().

We simply want to check if we have the .zip file is available and if the file can be found in the .zip file. If both conditions are true, we can indicate we found the file. Otherwise, we let the default implementation take over and perform any checks needed.

        public override bool FileExists(string virtualPath)
        {
            if (IsViewsZipFound && FileExistsInZip(virtualPath))
            {
                return true;
            }
            else
            {
                return base.FileExists(virtualPath);
            }
        }

        private bool IsViewsZipFound
        {
            get
            {
                if (File.Exists(ViewsZipPath))
                {
                    return true;
                }
                else
                {
                    return false;
                }
            }
        }

        private static string ViewsZipPath
        {
            get
            {
                return HttpContext.Current.Server.MapPath ( "~/Views.zip" );
            }
        }

        private static string TransformVirtualPath2ZipPath(string virtualPath)
        {
            string fullPath = virtualPath;
            if (fullPath.StartsWith("~/"))
            {
                fullPath = fullPath.Substring(2);
            }
            if (fullPath.StartsWith("/"))
            {
                fullPath = fullPath.Substring(1);
            }
            return fullPath;
        }

        private bool FileExistsInZip(string virtualPath)
        {
            string zipPath = TransformVirtualPath2ZipPath(virtualPath);

            using (ZipInputStream s = new ZipInputStream(File.OpenRead(ViewsZipPath)))
            {
    			ZipEntry theEntry;
                while ((theEntry = s.GetNextEntry()) != null)
                {
                    if (theEntry.IsFile && theEntry.Name == zipPath)
                    {
                        return true;
                    }
                }
            }

            return false;
        }

So to check if the .zip exists, we just use the System.IO.File.Exists() method. To verify if the file exists, we read the .zip file entries and try to find one that matches our file. If we find it, we return true. Simple enough.

For getting the file, we use similar code:

        public override VirtualFile GetFile(string virtualPath)
        {
            if (IsViewsZipFound)
            {
                var file = ExtractFromZip(virtualPath);
                if (file != null)
                {
                    return file;
                }
            }

            return base.GetFile(virtualPath);
        }

        private VirtualFile ExtractFromZip (string virtualPath)
        {
            string zipPath = TransformVirtualPath2ZipPath(virtualPath);

            using (ZipInputStream s = new ZipInputStream(File.OpenRead(ViewsZipPath)))
            {
                ZipEntry theEntry;
                while ((theEntry = s.GetNextEntry()) != null)
                {
                    if (theEntry.IsFile && theEntry.Name == zipPath)
                    {
                        byte[] data = new byte[theEntry.Size];
                        s.Read(data, 0, data.Length);
                        return new ZippedVirtualFile(virtualPath, data);
                    }
                }
            }

            return null;
        }

Here we check if the .zip file exists, and then try to extract the file from it. If it is found, it is also returned. A new class called ZippedVirtualFile is used to return the file, this will be introduced a little later below.

Finally, we override the GetCacheDependency() method, because MVC uses that to monitor when it needs to rebuild/reread a file. What we will do is instead of returning a cache dependency on the file, we return a dependency on the .zip file. If the .zip file changes we will assume our file changes. Not very robust, but will work for this example.

        public override System.Web.Caching.CacheDependency GetCacheDependency(string virtualPath, System.Collections.IEnumerable virtualPathDependencies, DateTime utcStart)
        {
            if (IsViewsZipFound && FileExistsInZip(virtualPath))
            {
                return new System.Web.Caching.CacheDependency(ViewsZipPath, utcStart); 
            }
            else
            {
                return base.GetCacheDependency(virtualPath, virtualPathDependencies, utcStart);
            }
        }

ZippedVirtualFile

We need a little helper class that will hold our extracted file, and return a stream to it on demand.

    public class ZippedVirtualFile : VirtualFile
    {
        public byte[] data;

        public ZippedVirtualFile(string virtualPath, byte[] data)
            : base(virtualPath)
        {
            this.data = data;
        }

        public override System.IO.Stream Open()
        {
            return new MemoryStream(data);
        }
    }

The class is very simple, it simply holds a byte array, and when a stream is requested, returns a new MemoryStream that can be used to read the contents.


Using the ZippedVirtualPathProvider

As a final step, we need to register our provider. This can be done in global.asax.cs by adding the following code to the Application_Start() method:

        protected void Application_Start()
        {
            // ...

            ZippedVirtualPathProvider.RegisterMe();
        }

And we are ready to go! (As soon as we create the Views.zip file of course).

I was trying to find a solution to a problem with LINQ and its CreateDatabase() call (see details in another blog post). Then I bumped into the following problem.

When I tried attaching a database into an SQL Server Express User Instance that existed before (the same database under the same path), I received an exception. I actually manually deleted the .MDF and .LDF files from disc, and then tried to recreate them programatically. The exception informed me the database file already existed. Although I have deleted it myself.

The exception said: Database 'path_to_database' already exists. Choose a different database name.. Which was very strange.

Reading more about SQL Server Express User Instances, I found out that they are more complicated than I thought. What actually happens is that SQL Server Express copies the master and msdb databases under the user's directory (who runs the user instance). It then starts the user instance (sqlservr.exe) under the user account. When you want to use a database in a user instance, it actually attaches the database file to the user instance - just as you would attach a database to a regular SQL Server instance. It just happens behind the scenes.

This creates trouble of course, because the database is registered somewhere, and you shouldn't just delete the files. Which I did. And that is why I got the error message.

There is some information about a tool that you can use to connect to a user instance and execute commands (detach maybe?). I found it too complicated. (But if you are interested, visit this MSDN page for more information!)

Rather, I killed my user instance process (Task Manager, kill sqlservr.exe). Then proceeded to delete my user instance directory. For Windows 7, this would be:

c:\Users\username\AppData\Local\Microsoft\Microsoft SQL Server Data\SQLEXPRESS\

I then restarted my application (which called CreateDatabase()), and it worked. It recreated the above directory, and everything was fine.

I have to admit that this seems like a bit of a drastic solution, but it sure is faster than connecting to the instance and issuing SQL commands by hand

And it should not have any side effects, because all user instance databases will just get attached automatically the next time you use them.

You might be familiar with SQL Server Express User Instances. These are per user copies of the database server, into which you can attach files "on the fly", without a real need for a fully managed SQL environment. You probably bumped into it if you have used the default setup for MVC or ASP.NET where you tried to use membership or other services. ASP.NET created a .MDF and .LDF file for you in the App_Data directory automatically. These are actually SQL Server Express User Instance databases.

If you ever checked the ConnectionString, it will contain a |DataDirectory| directive (and the term User intance=true. At runtime, the ASP.NET runtime resolves this to point into the App_Data folder of the application. The SQL client runtime of .NET is able to connect up this file into the user instance and use it, without any user action needed. It can also create it, if it does not exist.

While ASP.NET provides this behavior for its own databases by default, there is nothing stopping us from using the same feature with our custom application databases.


Create a missing database

When I put code into source code control, I rarely put it actual .MDF and .LDF files (databases). Instead, usually an .SQL script goes in there that can create my database. However, since LINQ has the neat feature to create the missing database, why not use that? This way I could make my project create the database it needs by itself, when starting up.

It would be ideal for distributing demos and examples. The person who opens it up, just needs to start it. Visual Studio self hosts the web application in its own web server, the database is created in the SQL Server Express User Instance, and it just starts working.

It is all very simple in theory, until you want to call LINQ's CreateDatabase() on a LINQ model that has a ConnectionString with the |DataDirectory| variable. I used CreateDatabase() succesfully before with regular SQL server databases. But when I tried the user instance approached, it failed.

My first try was the following code:

MyDataContext ctx = new MyDataContext( connString )
if (ctx.DatabaseExists() == false)
{
    ctx.CreateDatabase();
}

And what I got was an ArgumentException, that told me: Illegal characters in path.. Strange. I checked the callstack, which was:

   at System.IO.Path.CheckInvalidPathChars(String path)
   at System.IO.Path.NormalizePath(String path, Boolean fullCheck, Int32 maxPathLength)
   at System.IO.Path.GetFullPathInternal(String path)
   at System.IO.Path.GetFullPath(String path)
   at System.Data.Linq.SqlClient.SqlProvider.System.Data.Linq.Provider.IProvider.CreateDatabase()
   at System.Data.Linq.DataContext.CreateDatabase()

I checked everything, but all seemed to be fine.

I then tried substituting the actual path in the connection string, and my code worked. So what is going on?


Bug in LINQ DataContext

LINQ DataContext uses its own SQL wrapper, and that wrapper has a bug. It cannot resolve the |DataDirectory| constant. I used to .NET Reflector to verify this. Sure enough, the System.Data.Linq.SqlClient.SqlProvider class retrieves the AttachDbFilename entry from the connection string, and then uses it directly. So if you have |DataDirectory| in there, you are out of luck: an exception will inform you of this

Luckily, you can retrieve the value of the DataDirectory entry from the current AppDomain. So you could do the substitution yourself, pass the corrected connection string to the DataContext and create the database. Voila!


DataDirectory

So first, we need the DataDirectory entry. We can use the following code to retrieve it from the current AppDomain:

string dataDirectory = AppDomain.CurrentDomain.GetData("DataDirectory").ToString();

Putting it together

So I ended up writing the following method into my Global.asax.cs file:

        private void EnsureDatabase()
        {
            string connString = null;
            connString = global::System.Configuration.ConfigurationManager.ConnectionStrings["MyConnectionString"].ConnectionString;

            if (connString.Contains("|DataDirectory|"))
            {
                string dataDirectory = AppDomain.CurrentDomain.GetData("DataDirectory").ToString();
                connString = connString.Replace("|DataDirectory|", dataDirectory);
            } 

            using ( MyDataContext ctx = new MyDataContext( connString ) )
            {
                if (ctx.DatabaseExists() == false)
                {
                    ctx.CreateDatabase();
                }
            }
        }

This code loads my connection string and if it finds the |DataDirectory| directive in there, it will replace it with the DataDirectory value from the AppDomain. It then fires up an instance of the DataContext and makes sure the database exists. Sure enough, this works (except ... see below).

This code can be called in Global.asax.cs in the Application_Start() method. After this, whenever you run the application and it finds the database is missing, it will create it. You would still need to populate it with initial data, if that is a requirement, because the database will be empty of course.


Multiple user instance databases

While testing the above problem, I replace the DataDirectory variable by hand first, tried it that way, and it worked. I then stopped the web server, deleted the created files and started playing around. I bumped into a problem when I then wanted to create the files again. As if SQL server believed the files existed, although I deleted them. As it turns out, User Instances are not as simple as they sound. There is a slight problem when you want to attach a database with the same path into the user instance that already existed. I wrote a separate blog post about that. See it here.

I find it interesting that some common usage scenarios are missing from both LINQ to SQL and the Entity Framework. I did some research on the internet, and I do not seem to be the only person who is having these problems. There are solutions to some, but I still feel the need to rant about these in this blog

Creating the database from the model using code

The first problem is creating the database from the model in the ORM, from code. By the term "from code" I mean creating the model when the application is running instead of using design time tools in Visual Studio.

LINQ to SQL supports this scenario by providing you with a CreateDatabase() and DeleteDatabase() method, that will essentially create the database and destroy it. It works as expected, but is not enough. There are some limitations as well, but I could live with them. For example, the LINQ model does not describe every piece of metadata that the original table would have (from which the model was generated). So when the model is used to generate the database, some info will be missing, like users, usage rights, stored procedures, etc. As said, this would not be a problem if all I need is the tables themselves.

But there is no fine grained control over the creation process! The CreateDatabase() method will either

a) create the entire database from scratch
b) or FAIL

Now what I would want to do is be able to create my tables independent of the database. If the database exists, just add my tables into it. LINQ to SQL does not seem to be able to support this scenario. There is an awful lot of support built into the LINQ to SQL classes for generating all the schema etc. (Reflector shows them … they are of course internal to LINQ to SQL). But you cannot create just the tables. Or just the missing tables.

Now why would I want something like that? Let me put up a few scenarios:

a) In a shared hosting environment you usually have 1 database. So the database is shared between different applications. If I have two applications that I would want to be able to dynamically create the tables, the second one to be run is just out of luck if wants to use this method.
b) If I have an application that is partitioned into components and components can be plugged in dynamically, then I would like to have all components having their own LINQ mappings. Thus the situation is similar to the one described above: two or more sets of tables need to be created. I would even support the opposite, when a component is uninstalled, its tables get removed.

The Entity Framework has a similar CreateDatabase() method, with the same faulty assumptions the LINQ version makes: if the database already exists, it will just throw an exception. There is an interesting method in EF though, CreateDatabaseScript(). This will generate the script that - according to MSDN - the CreateDatabase() method will execute. HOWEVER, the script this method returns omits the database creation statements. So this could be used to create the tables.

Otherwise for now the best solution seems to be to generate the database script files by hand, include them in the project and run them using SqlConnection/SqlCommand. This way I can be sure of what happens, and no existing tables are damaged.


Table names (prefixes, postfixes, entire names)

Sometimes the names of the tables will change during runtime. It can be a complete change or it can be something like a prefix that is applied to the table name. There can be a million reasons for this, here are two of those:

a) In a shared hosting environment you will usually get only one database. If you wish to use different applications you could run into naming collisions without prefixes.
b) You are writing a component that will be used by other programs. To play nicely along, your component should not “lock” into using a given table name. Prefixes or changing names should be supported.

It is dreadful how difficult it is to change table names at runtime for both LINQ to SQL and the Entity Framework. I found posts that describe ways to do it, but the entire process is complicated and difficult to automate. Especially the EF way seems very tricky.

Nevertheless, solutions do exist.

The following article describes a way for both technologies. For LINQ to SQL, you need to generate a mapping file and use it at runtime. What this means, that you cannot automate the process, and that you need to generate and modify the file again, should your database or mapping change.

For the EF scenario it also requires file changes, which is not too good. Anyway, here it is:

http://theruntime.com/blogs/jacob/archive/2008/08/27/changing-table-names-in-an-orm.aspx

For EF there is a better way, a small framework found on CodePlex. It enables you to play with metadata at runtime, like adding a table prefix. This solves the problem, but it also requires some code modifications that will disappear (and need to be redone), should you use the designer.

http://efmodeladapter.codeplex.com/

I cannot understand how these feature can be missing, especially from EF 4, which should be a mature product (I know LINQ to SQL is not being actively developed further). Even Microsoft’s own ASP.NET tables use the prefix notation to make their tables stand out. Again, in shared hosting environments it might be good to have prefixes. Versioning scenarios could involve prefixes or postfixes (== table name changes). Use cases are virtually endless, and it is a pretty straightforward thing to implement.

Summary

Creating tables from models and changing names after compile time seem to be simple problems for me - I would even call them everyday problems. And still they are not solved in Microsofts own ORM tools.

I would love to hear feedback on this post, feel free to suggest other solutions, to point out errors or to comment on these things. Also, experience with other OR mappers is welcome.

Creating WCF servics is really very easy with Visual Studio. All the basic code is generated for you when you add a service to an existing web project or start a new WCF Service Application project. Modifying the "contract" is pretty straightforward too: you just modify the interface for your service, and then the actual implementation. WCF will do everything for you, generate the WSDL, etc. All your clients need to do is download/refresh the WSDL and they are able to use your service. Easy, right?

But when you look at the WSDL that gets generated for your service, you will notice that there are a lot of references to an XML namespace http://tempuri.org. Lets just say if you publish such a service or develope a solution for yor client who will then publish the service with this namespace to their customers - it is just very unprofessional.

Changing the namespace, however, can be more trickier than you thought.

The following article will show you how to deal with namespace changes in WCF. This will work in both WCF 3 / 3.5 and WCF 4. There are some minor changes, for example, with the configuration, but the same things apply.


Quick reference

For those who are impatient to see how this is done, here is a quick reference of the changes we will apply to a simple WCF service to change the namespace. There are four places to change:

- Apply the Namespace property to the ServiceContractAttribute on the service contract interface
- Apply the Namespace property to the ServiceBehaviorAttribute on the class that implements the service
- Apply the Namespace property to the DataContractAttribute on every class that is involved with the service (parameters, return values)
- Change the namespace for the binding, either on the binding class or in the .config file

And now in detail.


The contract namespace

The first step of course is changing the contract namespace. This is pretty easy, and probably something everyone tries first. It is however, not enough. But lets get started anyway.

Assume I have just generated a new WCF service application. I renamed the default Service1 name to RebuildallService, so I have an interface called IRebuildallService and a class called RebuildallService. The .svc file is called RebuildallService.svc. A little too many service words, but I will leave it that way for now

The service contract looks like this:

    [ServiceContract]
    public interface IRebuildallService
    {

The ServiceContractAttribute accepts a property called Namespace, and you might have guessed we are going to use that. This will change the namespace on the contract.

So the code now looks lke this:

    [ServiceContract ( Namespace = "http://schemas.umbraworks.net/rebuildall" )]
    public interface IRebuildallService
    {

If I compile and check the WSDL, I will get certain parts with the tempuri.org namespace and others parts with my new namespace. In practice, the wsdl:definitions XML elements will be in the new namespace, but all data and other parts of the WSDL will stil exist in tempuri.org.

A note on namespaces: namespaces ARE NOT URLS!. They might look like one, like in my examples, but there is no such subdomain as schemas actually in existence. Namespaces follow the URI format, but are not actual addresses. They can be used to identify schemas, because usually a company owns a domain name. Thus using that as the schema namespace creates something unique. And that is exactly what namespaces should be: unique.


The service namespace

So, what else do we need to change? The next step is of course the service implementation itself. Currently, it looks like this:

    public class RebuildallService : IRebuildallService
    {

What we want to do is apply the ServiceBehaviorAttribute, which also has a Namespace property.

    [ServiceBehavior(Namespace = "http://schemas.umbraworks.net/rebuildall")]
    public class RebuildallService : IRebuildallService
    {

Looking at the WSDL after this change you will notice that a big part is now in the desired namespace and there are two things left in the tempuri.org namespace: the data types we use and the wsdl:binding element.


The data types

Lets change the namespace for the data types, since it is a rather straightforward change. Currently the example service (the default implementation that Visual Studio provides) contains the following definition for a data type:

    [DataContract]
    public class CompositeType
    {

You might have guessed, we will add the Namespace attribute to the DataContractAttribute.

    [DataContract(Namespace = "http://schemas.umbraworks.net/rebuildall")]
    public class CompositeType
    {

This will change our data types to exist in our new namespace.

If your data is more complex and you have several classes, be sure to apply the Namespace property along with the DataContract attribute to every one of them. Do this also for your enums, otherwise they will exist in the tempuri.org namespace!


The binding

The final element is changing the binding namespace. If you use code-only WCF, you would specify this in the constructor of the Binding class, but since most of the time you will use config files to specify the binding, I will show how to proceed in this latter case.

In WCF4 you have default bindings, so your web.config file will be pretty empty. In WCF3/3.5 the config file will contain the binding by default, so there you can change it more easily.

Anyhow, what you need to do is add the bindingNamespace element to the endpoint element of the service element. Like so:

    <services>
      <service name="WcfNamespaces.RebuildallService">
        <endpoint address="RebuildallService.svc" bindingNamespace="http://schemas.umbraworks.net/rebuildall"
                  binding="basicHttpBinding" contract="WcfNamespaces.IRebuildallService" />
      </service>
    </services>

For WCF4, I would need to add this entire section into my config file. For WCF3/3.5 only the new attribute needs to be added. The WcfNamespaces, in case you wonder, is the .NET namespace my test project was in.

And with this final change, the WSDL will not be in our own custom namespace, and http://tempuri.org is gone for good

Some thoughts about namespaces

In the example above, I placed all services, descriptions, bindings and data types in the same namespace. This is of course not needed, you can specify different namespaces for everything, and WCF will handle this. The WSDL will be more fragmented: WCF generates a separate document for different namespaced entities. But there is nothing technically preventing you from doing this.