Web – Responsible Coder

Remake of cascading dropdowns with KnockoutJS: Part 1 – creation of custom bindings

One of all-time most popular articles on this blog is AJAX Cascading Dropdown (actually whole series), which still receives lot of attention even after 3 years.

Not surprising, as this is universal problem that is always present, and any business has some kinds of categories and groups of its own, with more or less levels deep.

Small aside to prevent wasting your time: if you are advanced KnockoutJS user, then you might want to wait for next article of this series, where I will be improving binding from this one to add cascading interactions. This post is more detailed and suited for developers who had only used default bindings that come with KnockoutJS, without many customization.

As browsers advanced, and got closer and closer to a common standard, jQuery became less and less necessary, and other frameworks emerged to address problems of a mess that very often gets produced when many developers try to produce “rich client” UI with jQuery. Hard-coded ids, storing variables in hidden fields to communicate between client and server (and even to communicate between different JavaScript “modules” when there is no real modularity and dependency injection), global functions and variables that are shared between in-line JavaScript and referenced files… you name it. Don’t get me wrong, I am aware that it IS possible to produce something clean and maintainable using jQuery and its plugins, and if you happen to be one of these people who knows how to do it, just think about how easy is to find team of developers who know the same? And even if only benefit of MVVM and data-binding frameworks would be avoiding of copy-pasting selectors all over the place and removing hidden fields, it would be worth learning them, so it is not surprise that they are now so popular for client-side development in JavaScript.

So, these two facts, popularity of my first post about jQuery version of cascading dropdowns, and popularity of Knockout are obviously good clue for me that people are solving same problem today with Knockout, and this is why I decided to create new, updated version of my initial blog post, this time with KnockoutJS. For any of you asking why KnockoutJS and not some other library, my answer is that is first data-binding library I learned couple years back, and even if it is not my favorite anymore, I still like it, it is being used on most of projects I work on, and I believe it won’t go away just yet, so with many people using it and continuing to use it in future, I believe that this article will provide quality content for my readers. And, besides that, I’ll make AngularJS version next :).

So, goal of this series of articles is to implement support for series of dropdowns that are depending on each other (choices avalable in some of them depend in chosen values in others), with items being loaded from remote service, and dependencies between them being managed automatically. All of that should be simple to use and reusable, meaning that no new code needs to be written when you use it 5, 10 or 50 times. To make that work, I will create custom binding. If you never created custom binding, I hope that this article will give you all necessary guidance to be able to create other bindings, depending on your needs.

In this, first article, I’m going to make step-by-step guide on how to implement part of this feature as custom knockout binding, which loads items from server, but without cascading behavior (so multiple dropdowns, no interactions between them), and at the end you have link to source code if you just want to quickly add this binding to your project and (hopefully) solve at least one of your problems. By reading it, you will also find out about my approach to creation of custom bindings. I derived it after years of experience in creating them, and I believe it is good, as it allows you to add one small tweak at a time, and you always know if a line you added breaks something. Obviously, there are always areas to improve, and if you have some tips, feel free to share them in comments. Custom bindings tend to be pieces of code you don’t write every day, so you usually forget how they work, and what are arguments for handlers. Therefore, taking this iterative approach is what I usually do when I see that I need new binding, if I haven’t created one yesterday (so that I remember all this stuff and can skip baby steps).

If you don’t want to go through all these steps manually, you can download finished code sample produced in this post from this changeset on GitHub and fiddle with it while reading this post.

To reduce overhead for getting some base to work on, I will use ASP.NET MVC5 with Bootstrap 3.1.1 LESS template for Visual Studio. This template gives you already working application with decent css where I can start implementing this (almost) immediately. Only missing prerequisites that I need to add are knockout and ko lite tools:

PM> Install-Package knockoutjs
PM> Install-Package KoLite

To start with, I’m cleaning up index page, adding reference to knockout and activity plug-in from ko lite tools (that we don’t need yet, but when options are loaded with ajax, some form of feedback for user will be necessary), and creating one select to bind with some options:

Screenshot 2014-05-11-16-37-33-8117064

If you didn’t know, you can hardcode your data when using knockoutjs bindings, and model is here just because I plan to use it, but this already produces dropdown with two options, second being selected:

Now, on to create custom binding. As I want to display some options, that will be loaded using ajax, good place to start is default ‘options’ binding that comes with knockout. For start, you can create your alias for it (pass through call to options binding without changing/adding anything), and then run application to make sure all still works as before. So, I created ‘remoteOptions’ binding and used it instead of ‘options’ to check if all works as expected:

If you don’t know how bindings in Knockout work, here is very short explanation for arguments that are passed to your handler:

element is DOM element of page that contains data-bind attribute for your binding,
valueAccessor is function that returns object assigned to your binding in data-bind attribute,
allBindings is an object that contains all parsed bindings from data-bind attribute.
bindingContext an object that contains and $data property that represents your current model (i.e. model that you used in applyBindings, or current item in foreach binding, or property that was used in with binding), but also $parent, $parents and $root properties where you can access other parts of your model (traverse up from current node)

For more details, read documentation.

So, my goal is to tell my binding where to load options from and avoid having either options or remote address in my view model. That reduces number of properties and lists in view model, and in many cases has effect of making view model much more like (data) model reducing mapping complexity, or removing mapping completely. So, having that in mind, first step will be to change binding syntax (I also changed html and added some css but that is just markup, important part is in red):

This now passes an object instead of array of options, and to avoid ko options binding being broken with it, I’m changing my custom binding, just a little bit:

Screenshot 2014-05-11-20-37-08-8765880

As you can see, I’m reading new binding by calling valueAccessor() and at the moment doing nothing except logging its content to console to show you what is in:

{“url”:”categories/get”}

At same time, I created function optionsValueAccessor (actually just moved hard-coded options here) to provide some options that default binding will read and use to display options in select element:

I added another option now, and it can be seen in page:

Ok, that part seems to be simple enough, now to get some real options using ajax. For that, I’m exposing very simple http service in web app to return contents of an enum (in real app you would probably read that from some kind of persistence service):

So, I want to load list from this http service automatically as options in Category dropdown when page is loaded (and binding initialized), and to accomplish that, I want to have minimal amount of ~~coding~~ configuration, and that would be address of endpoint that contains options for dropdown:

remoteOptions:{ url:’/categories/get’ }

That is rather easy to accomplish with knockout, as we can pass reference to an observable array to underlying default options binding which will show options from that observable array, and set its items once ajax request has been completed. Single problem is that we have to make that ajax call in init method of bindingHandler, as they need to be loaded only once, update is triggered every time user selects item from dropdown, and every time update is triggered, we need to pass same observable array to underlying options binding (no need to refresh all items when selecting one of them). I noticed that passing reference from init to update handlers is not simple, as you can’t write to allBindings (well, you can, but it doesn’t get persisted anywhere, next time when update is called, you get new object without your changes), and you cannot write to value of your property in model, as it is supposed to hold value user has selected (having something else defeats purpose of this blog post). Fortunately, value binds to observable, which is a function, and in JavaScript everything is object (including function), so we can simply add a property to that object (observable, function) that will hold reference to observable array with options:

To reduce probability of interference with something else that may be in observable function/object, I tried to give unique name to my property: ‘__remoteDropdown_options’ (if I used something like ‘options’ here, somebody else might used same property name for something else and then if both bindings are used at same time they would read/write different thing in/from same place, and there might be very weird and hard to track down bugs). As you can see, I’m setting it to observable array in init, and firing off an ajax request to load options from url passed in binding. Reference to that same options observable array is returned as result of optionsValueAccessor in update function that initializes dropdown. When ajax request completes, options is filled with result from server, and voila, my dropdown contains these values:

That is kinda cool (especially because with these 10 lines of code you can fill in any dropdown with ajax), but, there is still no trace of “cascading” from title of this post. And, not to mention that in most cases description shown in dropdown will differ from value (i.e. you will have item’s primary key as value, and description as text shown to user). So, let’s add that before trying to add cascading behavior (because that is short and easy, and this post already becomes very long, too long for binding of 10 lines).

First, instead of list of strings, return list of objects that have Id and Text:

That makes controller now return array like this:

But dropdown now shows three [object Object] items instead of text. Fortunately, we are passing through allBindings untouched to knockout’s default options binding, which already has support for this (read documentation for details). To use this feature, I just need to tell knockout what is name of my text property: optionsText:’Text’. Now is right time to add category as property on my view model to be able to show what is selected:

And yes, that is view model. I know that some of you are going to have hard time to accept that I did not define view model in C#, but it is intentional, not only for this sample, it can be avoided completely. You have view model here, and I believe one is enough. Keep it simple. You don’t need it even when you are accepting your domain entity in controller (when you are saving it), you can define model binder to initialize everything that you need, effectively avoiding one layer of view model classes and mappers (mapper does the same your model binder would do, but you need to add code to call it into your controller, and why do that when MVC already has extensibility point for that?) This is a topic for separate post (still in drafts folder), but feel free to comment about it here.

So, with this here, I’m changing markup to reflect latest change in options that are returned from server, and with having observable to store category in view model, I’m going to show it on screen:

As you can see, when using remoteOptions, you can use optionsText as with default options binding to choose property to display in dropdown, and value (selected item from dropdown) is bound to category.

As I did not set optionsValue, knockout will set whole object (with both Id and Text) into category observable, and you can see that from with binding and results below:

These are 3 screenshots of same dropdown with different option selected, as you see, it automatically shows chosen category below.

So, to conclude this first part, I’m going to add another dropdown, with subcategories (that should be filtered by category selected in first dropdown, but not yet). For that I need another http service to get data from, and another html markup for dropdown:

So, for this dropdown, different label, different url, and different property in vew model (I have added subcategory: ko.observable()). That produces:

Screenshot 2014-05-12-11-36-51-8104404

With this, I’m ending this post, and source code changeset on GitHub is exactly the place where I’m starting next post, in which I will add dependencies between these dropdowns and filter items in “children” based on selected items in “parents”.

P.S. One of additional purposes of this article is also to document/explain feature that I implemented on my latest project, binding which works on same principles as this, and anyone having trouble understanding it can come here for help. Even if I’m sad that my code requires additional explanation, it is still a good way to celebrate successful migration to new domain for my blog.

Nancy on OWIN – bundling and minification

Continuing with changeset from last time.

I want to introduce bundling and minification without dependency on System.Web (it would defeat purpose of trying to create site on OWIN).

For that, I’m using SquishIt:

Install-Package SquishIt

I found some resources how to use it in Asp.Net Mvc and there is also integration manual on Nancy documentation, none of them is covering case when Nancy is running outside of IIS (and without Razor engine, which I don’t believe it is necessary for this simple example, I might want to add it if I see that I need it, but I want to try to go without it), but they are good samples to see how to use SquishIt API. For bundling, I need bootstraper, which is equivalent of Global.asax.cs in Asp.Net application, and as I want to do minimal customizations, I will use default bootstrapper as base for mine:

I’m taking care of bundling itself in separate class, BundleConfig, as Bootstapper is place where all of initialization is performed, and if all that stuff is inside it will become very dirty and cluttered in notime:

I’m separating JavaScript and Css bundling, and within JavaScript, creating one bundle for all libs (and only angular is in it), and another for my “app” where I put whatever js is in app folder.

To keep things simple, and continue using html view, which can only replace tokens from passed view model, I’m going to reference these bundles and pass them to view from my HomeModule:

That produces already complete html tags, so that I can only write them directly in view:

Result:

As a side note, take a good look on contents of this file. You can see that WelcomeCtrl was minified, and minifier used (MsMinifier) did shortening of controller dependencies (which was just ‘$scope’ in this case). Take care when writing applications in Angular to explicitly identify dependencies (by defining modules in array, with dependency names as strings first, then your module as last element of array, like here, “$scope” is a string, and minification does not change it). That allows Angular’s injector service to identify your dependencies and inject them even when minifier performs obfuscation of argument names to reduce amount of characters used.

SquishIt works by taking all of files added to bundle, and packing them together into another file, that will have given name. If you want to leverage browser caching, then you need to add “#” to file name, and it will be replaced with a “token”, that looked to me as hash at first, but I figured that it must be just a random guid, as it did recreate scripts combined js file without changing any of contents, so it just creates a file with random guid in name and remembers it under given name for lifecycle of application instance.

That is very simple approach, as then all that need to be done is render include tag that references that file name as source, and when something is changed, new file name and url will be created, so no outdated files will be used, and no browser cache will cause problems. It seems very simple and effective, but don’t do it like this! This is just a step to learn how it works, and while it seems fine, it is not the simplest thing that could possibly work. It will create new file every time application is restarted, and if you are hosting your application, you may not be able to control app pool life cycle, which may have rather conservative life policy. If your application works good and you don’t have to check on it, it may generate hundreds of hundreds of these minified files. Another, even more dangerous problem is that I placed my minified scripts file into same directory with other files, while at same time including whole that directory into bundle! That means every time when app pool is recycled, new bundle will be created with all files, and previous bundle(s), effectively doubling size of app folder on every start of application. This exponential progression can go undetected while doing test run of application, because angular tends to be pretty resilient, and even if you redefined some modules few times, it will work, but after couple dozens of restarts, size of served JavaScript can grow to megabytes, effectively killing browser, and eating your outgoing bandwidth on server. And taking your hard drive space. All of that can be pretty expensive. You understand now why I wrote don’t do it in red, it is very dangerous to use it, even if you put your minified file into another folder, you can forget, or someone else who needs to change something will see how bundles are defined and put it there not knowing how it works, or your hosting provider might decide to migrate your web site to new server and reset permissions in process, so you end up with:

What is another reason to avoid it, is that while compilation debug=”true” is set in web.config, SquishIt works same as standard bundling from Microsoft.Optimization, and will not do any minification, it will serve resources unchanged, as in manual version from start of this post, with only difference being that you don’t have to manually include every file from app folder. That makes developer life much easier, but if you don’t deliver version every day (it is as easy as defining publish profile and configuring server for web publishing, and publish in release mode), it will hide this problem until very late in development life cycle. You may figure out something is wrong if you disable debug and then re-enable when you see minified version being picked up too:

So, I hope that by now I convinced every single reader that read to this point, that this approach should not be used for serious application. Ever. This can be avoided by using caching of minified resources in memory. It is faster, doesn’t cause hit to hard drive on every request, it does not require write permissions for application pool account, and does not leave any files behind that must be cleaned up. There is already example in Nancy doc, under Advanced – Diskless Caching.

Based on that example, I created my BundlesModule:

This module is working from /bundles subpath, and also uses GZip to compress outgoing stream, and adds etag and Cache-Control headers to leverage browser caching. In debug mode I intentionally reduced caching to 45 seconds, so even if you create bundle with .ForceRelease to enable minification in debug mode, it won’t be cached for long as you will probably be changing files.

To be able to use dynamic parameters type, I had to add reference to Microsoft.CSharp.

As name of bundle is now passed as part of url ({name} part after /bundles/js/), creation of bundles must be changed accordingly, as they were created with different names:

Bundle name is now encoded into url in place where BundleModule expects {name} parameter, so rendered url will point back to that module, and pass bundle name that needs to be served:

There is no more writing to disk, no need for permissions, and we have bundling solution that is as effective as one from Asp.Net Mvc.

Source code is at github, as usual. From here there are many things that need to be done to make one application, and I still have not decided what to do first. If you have suggestions, please leave a comment. Thanks!

Nancy on OWIN – serving static resources

For this post, I’m using same playwithowing project I created during last blog post, which is on github.

First thing that I want to do, is add some NuGets:

PM> Install-Package bootstrap
PM> Install-Package angularjs

These two are enough for now, I have both css and javascript files that I need on client now.

To use them in view, next step in Asp.Net would be to drag&drop them into view, css in <head> and scripts to bottom of <body>. VS produces proper tags for them:

If you run application now, you can see that all already works, everything loads just fine:

That is very nice, as by default OWIN doesn’t serve static resources (you must configure simple FileServer, which you can do anyway if you want to enable file browsing in some of your folders, but then add builder.UseNancy() line at end of Configure method, as Nancy handler is greedy and it will handle request before it gets to file server). But even without it, this works because of default conventions in Nancy, which are summed in first sentence on that page: stick stuff in /Content .. done.

As a side note, notice that everything is loaded within 200ms,

which is nice, especially when having in mind that average

human needs 300-400ms to perform an blink of eye 🙂

So, this makes again another very simple blog post, to summarize plot started by the title, there is nothing needed to do to serve static resources, if you use Nancy on OWIN. Nancy already takes care of it, same way as it does on ASP.NET, so no need to worry what is underneath.

As this works, now you can start programming 🙂 There is one thing that is from my experience very neglected and many developers don’t know or don’t use it, and that is <base> html tag. It is very useful, especially if you want to create Single-Page or Rich-Client applications, which tend to load lot of resources from JavaScript and don’t have access to Html helpers like Url.Content or Url.Action that you can see all over the place in views in Asp.Net MVC.

To start building some app, I need to add some client-side functionality. If I replace home.html with basic bootstrap template, and add ng-app directive to html tag, I can check if angular works:

And, as it shows ‘it works!’ in place where I have put curly brackets, I’m happy. As this is only static html, Visual Studio Page Inspector will render exactly same result as browser. Now, I’ll add application root to <base> tag, which is by default “/”, but if you ever hosted your application in IIS, and if it was not in root, but in some path (like localhost/myapp) then you know how important was to render all resources with @Url.Content helper, because all you resources would be broken otherwise. So, after I added <base href=’/’ /> and changed css and script links from relative to app-relative (removed ‘../’ prefix), I’m adding project root folder to local iis, into http://localhost/playwithowin app:

Loading project url in browser shows that OWIN works on IIS, that’s nice, but that also kills css and js resources loading, because they are loaded from root of app, which is now wrong, and it is common thing happening after you put app in virtual folder when you don’t use @Url.Content helpers. Now my angular doesn’t work properly anymore, because it tries to load it from http://localhost/Scripts/angular.js instead of http://localhost/playwithowin/Scripts/angular.js:

To fix this, I will fix path in <base> tag: <base href=”playwithowin” />

That makes everything fine again:

Nice thing about <base> tag is that doesn’t only fix these urls, but as we will see, also removes need to have ‘root’, ‘approot’ or whatever is the name that you use in global javascript namespace to inject that path in master/layout page and to make js code aware of it for needs of loading any static resources asynchronously. For me that was always one of ugliest things in JavaScript, and now you can just forget about it. Just use apprelative paths in your app, and point base tag to your app root, whatever is it.

Next problem is, you may deploy your app to different environments, which may be on same server, in different virtual folders, or you may even want to have multiple instances of application on same server. To automate this, I’ll send base path from server:

Nancy view can accept anonymous view model, and I assign base path of request url (that is “Nancy root”) to server-side view model.

To use it in view, just use Razor-like syntax in html file! Thanks to The Super Simple View Engine, which is by default used in Nancy, it will be replaced with server variable:

So, to summarize this post, so far, using Nancy on OWIN is not different than using it on Asp.Net, and for many cases, Nancy’s Super Simple view engine is more than enough for what you need. If you still need Razor, just install NuGet: RazorEngine.

Next step in creation of app would be adding some custom js and html files. I will add angular app, one view (welcome.html), controller (WelcomeCtrl), and use them in home.html (set app name in ng-app in html tag, add div which references new view, and add references to new js files):

This is now very simple functionality, but we already have 3 referenced js files. Angular automatically loads views, but js files must be loaded explicitly, and in average application, with well separated responsibilities, there could be hundreds of js files. That could be managed using some AMD framework, but that is one of aspects where Asp.Net rules with bundling & minification, and I like it. It reduces number of requests, and we can leverage caching because it allows to automatically expire cache by varying url. In next post I will be looking on ways how to achieve same thing here.

Source code for this version is pushed to github.

Using Nancy on OWIN

I was keeping my eye on OWIN for some time now (since Mark Rendle’s interview in DNR in 2011, when he complained about HttpRequest and HttpResponse being sealed in ASP.Net and spoke about OWIN as alternative). I like simplicity of this interface, and I also like that Microsoft listened to community and provided Katana, their implementation of it (so now you can also run OWIN apps on IIS). OWIN got to stage where it can be used for serious applications, there is already support for multiple frameworks, some of them being Nancy, Singalr and WebAPI, so you can now build pretty much anything on top of it, and being independent from IIS and all of its http modules, handlers etc has a lot of potential to perform better (given that you don’t need all of these features that asp.net provides, which is the case for most of applications). And those of you who tried testing your code dependent on System.Web, probably already know about OWIN. For just one good reason why to use OWIN, I would just like to quote one part of that interview:

the code’s maintainability is that less code between the TCP socket and your framework. It feels naturally like that’s a good thing and it’s just less code in your application, that’s less code to go wrong. If everything is handled through interfaces or delegates, then at any point in that pipeline you can inject something in a test environment; and as far as everything beyond that point is concern, that might as well have come from a TCP socket so you can create a dictionary of headers and you don’t have to patch in everything else.

quote from transcript on http://s3.amazonaws.com/dnr/dotnetrocks_0683_mark_rendle.pdf

I’m big fan of clean and simple code above all, and I hate additional layers, especially when they don’t provide any valuable abstractions (don’t get me wrong, ASP.Net provides many useful abstractions, and it is great framework, but sometimes there is just no need for all of that), sometimes you just don’t need all that infrastructure, actually I would say that most of applications don’t need it, especially today when more and more applications are running in browser, you just need as simple way as possible for your application to communicate to server.

So, as I always want to learn something and extend my skill set, I choose to put OWIN under my belt, as creating HTTP services which can be self-hosted (or in windows service, or in exe/scheduled task, or…) can come in pretty handy.

To get started with OWIN, I’m using Visual Studio 2013, it takes few simple steps:

This gives you choose template screen:

That gives new project that is (aside from plenty references) empty, it contains only web.config.

If you are not using Resharper, then you should :), but for this matter you can use some other extension to clean up unused references, that will leave your project with only System and System.Core. To get started with Nancy and Owin, use NuGet PM Console: Install-Package Nancy.Owin. That will also pull Nancy, as dependency. That puts Nancy, Owin, and Nancy.Owin into references. After that, project looks like this:

To run app, you need to host it somewhere. As most convenient way to develop is to run it from Visual Studio, it is necessary to add hosting adapter to host OWIN in IIS (or Express version that is used by Visual Studio):

Install-Package Microsoft.Owin.Host.SystemWeb

This adds Microsoft implementation of Open Web Interface (aka Project Katana) and hosting adapter which contains OwinHttpHandler used to handle http requests and pass them to OWIN. These are in Microsoft.Owin, and Microsoft.Owin.Host.SystemWeb libraries, respectfully.

After these few steps, it is possible to run this frame for application:

As this is not MVC application from VS template, this is expected, but it is very nice and descriptive exception, and it makes clear what is next step to do to get it working: it is necessary to either add OwinStartup attribute to assembly, or to create Startup class with Configuration method. As “Startup.Configuration” sounds explanatory, I will do that:

Startup class

Configuration method’s signature is documented in Katana documentation, but you don’t even have to open it, if you don’t make it like this, another exception will tell you to do it. This part is actually what is this whole post about. This single line of code in Configuration method is actually all you need to do to run Nancy on OWIN.

It is worth noting that IAppBuilder is not official part of OWIN interface, but part of Katana itself, and if you decide to run your app on something else, then it may be necessary to change way how application starts.

UseNancy extension method comes from Nancy.Owin package, and it is just wrapper around builder.Use:

If you take a look into NancyOwinHost, you will see that it is just an adapter to Owin infrastructure, similar to NancyHandler for Asp.Net, but difference is that handler gets to work with HttpContextBase (which is abstract, and not very simple to mock/fake), while NancyOwinHost gets Dictionary<string,object> which is how http request is represented in Owin. It is lower level of abstraction, meaning it is much closer to the metal, and if you don’t need all of the fancy stuff in HttpContext, then it is likely that this is going to perform better, with less resources.

Running application now gives default Nancy 404 response:

Ok, Nancy gets request, and as there is nothing to process it, returns not found.

To return something useful to client, we need a “controller”. In Nancy, that is Module:

This is how handler for get request for root path (“/”) is defined in Nancy. I like it, as to me is much more expressive than “Index” ActionResult.

As I’m returning view named “home”, I will create it in “Views” folder (standard convention, Nancy will look for it there):

And result is:

So, our web site in .NET without System.Web is live:

This is yet far away from being an application, but building one using Nancy and Owin is not so much different than using MVC and Asp.Net. It is always useful to learn other ways to do same thing, because that gives you another perspective and ability to see things you missed before.

I end this post here, and next step will be more about Nancy, and how to add some script and css resources and see how to serve them to browser, and what is different in OWIN than in Asp.Net, as standard bundling libraries reside on top of System.Web and won’t work here.

I pushed source code for playwithowin project to github. You will have to restore NuGet packages as only code is there, so don’t go offline until you build it once 🙂

UPDATE: Next post is now published