Sunday, October 2, 2011

Your first app with CouchDB and relax-net

This is a tutorial that will show you how to create a simple tagged gallery ASP.NET MVC web app using CouchDB as a data store and relax-net as data access layer. It requires that you understand basic concepts behind CouchDB and have knowledge of ASP.NET MVC.  I’ll be focusing mainly on CouchDB related code in the post, but you can download full source here.

Preparation

In order to start, you first have to have a CouchDB up and running. I suggest you grab the community edition of Couchbase Single Server for Windows. After installing and making sure that CouchbaseServer service is started, you can visit http://localhost:5984/_utils/index.html address to run the Futon – CouchDB management interface. Create new database named tagged_gallery.

Create new ASP.NET MVC 3 project and call it TaggedGallery. Download and reference relax-net. Unfortunately it’s not available as NuGet package, so either grab the zipped release or just build it from source.

What do we build?

A simple gallery that allows registered users to upload pictures that all visitors can browse. Each picture can have multiple tags associated with it. All tags related to currently visible pictures display on the side along with count of pictures with given tag. If you click a tag, you can browse all pictures with that tag.

image

Infrastructure

relax-net takes some design patterns after NHibernate and one of them is a concept of session. Session is something that caches your entities during its lifetime and takes care that when you get a certain entity twice, you’ll get the same object reference. As with NHibernate, the best usage in web application is session per request pattern, which means you open session on BeginRequest and close it (or “return” in relax-net) on EndRequest. We have to have some means of managing the session and accessing it. The SessionManager component ‘s purpose is just that.

public interface ISessionManager
{
    void SetConnection(Connection connection);
    
    void OpenSession(string database);
    void ReturnSession();
    Session GetCurrentSession();

    IRepository<TEntity> GetRepositoryFor<TEntity>() where TEntity : class;
    Query<TEntity> GetQueryFor<TEntity>(string design, string view, bool reduce) where TEntity : class;
}

SetConnection initializes the SessionManager by providing Connection which basically points to CouchDB server. OpenSession and ReturnSession are self explanatory. In terms of relax-net you don’t “close” the session, but rather return it to the pool. GetCurrentSession allows you to get an open session for current request from any point in your app having access to SessionManager. There are also helper methods GetRepositoryFor<TEntity> (which creates and initializes a relax-net repository for given entity type) and GetQueryFor<TEntity> (which creates and initializes a query – query is a wrapper over CouchDB view). Now for the implementation:

public class SessionManager : ISessionManager
{
    private IUnityContainer _container;
    
    public SessionManager(IUnityContainer container)
    {
        _container = container;
    }
    
    public void SetConnection(Connection connection)
    {
        Contract.Requires(connection != null, "Connection cannot be null");

        connection.Observers.Add(c => _container.Resolve<AuditObserver>());
        connection.Observers.Add(c => _container.Resolve<TypeAssigningObserver>());
        HttpContext.Current.Application["DbConnection"] = connection;
    }

    public Session GetCurrentSession()
    {
        return DoGetSession();
    }

    public void OpenSession(string database)
    {
        var connection = HttpContext.Current.Application["DbConnection"] as Connection;
        Contract.Assert(connection != null, "Connection was not set");
        var session = connection.CreateSession(database);
        HttpContext.Current.Items["DbSession"] = session;
    }

    public void ReturnSession()
    {
        var session = DoGetSession();
        session.Connection.ReturnSession(session);
    }

    private static Session DoGetSession()
    {
        var session = HttpContext.Current.Items["DbSession"] as Session;
        Contract.Assert(session != null, "Session was not opened");
        return session;
    }


    public IRepository<TEntity> GetRepositoryFor<TEntity>() where TEntity : class
    {
        var resolverOverride = new ParameterOverride("sx", DoGetSession());
        return _container.Resolve<IRepository<TEntity>>(resolverOverride);
    }


    public Query<TEntity> GetQueryFor<TEntity>(string design, string view, bool reduce) where TEntity : class
    {
        return new Query<TEntity>(DoGetSession(), design, view, reduce);
    }
}

As you can see the connection is stored in HttpApplication (as it is shared for all requests) and session is stored in HttpContext (as it is private to a request). As might also noticed I’m using Unity as dependency injection framework. You can grab it by NuGet.

All relax-net entities have to inherit from Document base class, which has some basic properties like Id and Revision. By default Id for new document is created in form of [lowercase type name]-[guid]. We will introduce one more layer over the Document class in order to additionally store easily accessible type name in all entities. It will also allow us to add some common behavior for all our entities if we need that later.

public class BaseEntity : Document
{
    public string Type { get; set; }
}

There’s also the IAuditable interface which will allow us to track who and when created or modified the entity.

public interface IAuditable
{
    DateTime? CreatedDate { get; set; }
    DateTime? LastModifiedDate { get; set; }

    Reference<User> CreatedBy { get; set; }
    Reference<User> LastModifiedBy { get; set; }
}

Reference<TEntity> is relax-net way to associate different entities. Using reference in this case you can traverse from one entity to another dotting your way through. The references are lazily loaded, which means first time when you access the reference, relax-net will request appropriate entity it from CouchDB.

We don’t want to worry about having to fill fields from IAuditable or BaseEntity by hand every time an entity is created. Fortunately we can create observers that will react to different entity lifetime events. In SessionManager the connection is initialized with two observers: AuditObserver and TypeAssigningObserver.

public class TypeAssigningObserver : IObserver
{
    public void AfterDelete(object entity, Document document)
    {
    }

    public void AfterLoad(object entity, Document document)
    {
    }

    public void AfterSave(object entity, Document document)
    {
    }

    public Disposition BeforeDelete(object entity, Document document)
    {
        return Disposition.Continue;
    }

    public Disposition BeforeSave(object entity, Document document)
    {
        var baseEntity = entity as BaseEntity;
        if (baseEntity != null)
        {
            baseEntity.Type = baseEntity.GetType().Name.ToLowerInvariant();
        }
        return Disposition.Continue;
    }
}

public class AuditObserver : IObserver
{
    private ISessionManager _sessionManager;
    
    public AuditObserver(ISessionManager sessionManager)
    {
        _sessionManager = sessionManager;
    }

    public void AfterDelete(object entity, Document document)
    {
    }

    public void AfterLoad(object entity, Document document)
    {
    }

    public void AfterSave(object entity, Document document)
    {
    }

    public Disposition BeforeDelete(object entity, Document document)
    {
        return Disposition.Continue;
    }

    public Disposition BeforeSave(object entity, Document document)
    {
        var auditable = entity as IAuditable;
        if (auditable != null)
        {
            // create lazy references instead of weak references to overcome problems while running under debbuger
            // no perf penalty
            
            // if revision is empty, this document is being created
            if (string.IsNullOrEmpty(document.Revision))
            {
                auditable.CreatedDate = DateTime.Now;
                auditable.CreatedBy = Reference.To<User>(_sessionManager.GetCurrentSession(), 
                    "user-" + HttpContext.Current.User.Identity.Name);
            }
            auditable.LastModifiedDate = DateTime.Now;
            auditable.LastModifiedBy = Reference.To<User>(_sessionManager.GetCurrentSession(),
                "user-" + HttpContext.Current.User.Identity.Name);
        }
        return Disposition.Continue;
    }
}

As you might already guessed we won’t be using a standard Id format for users, but rather something like user-[login].

Model and views

Our model will consist of two entities: Picture and User.

public class Picture : BaseEntity, IAuditable, IHasAttachments
{        
    [Required(ErrorMessage = "Title is required")]
    [StringLength(50, ErrorMessage = "Cannot be longer than 50 chars")]
    public string Title { get; set; }

    public string ContentType { get; set; }

    public List<string> Tags { get; set; }

    public DateTime? CreatedDate { get; set; }
    public DateTime? LastModifiedDate { get; set; }
    public Reference<User> CreatedBy { get; set; }
    public Reference<User> LastModifiedBy { get; set; }

    public Attachments Attachments { get; set; }
}

public class User : BaseEntity
{
    public string Login { get; set; }
    public string Email { get; set; }
    public string PasswordHash { get; set; }
}

As you can see, Picture has a collection of tags, but tag is not a separate entity as it would be in normalized relational model. Tag collection will be stored within picture document. The IHasAttachements is mandatory for relax-net entities, that have binary attachments associated with them (this scenario is supported by CouchDB).

There are also additional classes used to provide a strongly typed model for the views.

public class TagWithCount
{
    public string Tag { get; set; }
    public int Count { get; set; }
}

public class SinglePictureModel
{
    public Picture Picture { get; set; }
    public TagWithCount[] Tags { get; set; }
}

public class PictureListModel
{
    public Picture[] Pictures { get; set; }
    public TagWithCount[] Tags { get; set; }
}

To query our database efficiently and get required data we have to implement some views. Views will be stored in _design/picture design document, which will help relax-net to load them basing on entity type name. You can edit the views using Futon. First of them named by-date will allow us to list all pictures in database ordered by creation date:

// by-date

// map
function (doc) {
    if (doc.Type == 'picture')
        emit(doc.CreatedDate, doc._id);
}

This view returns CreatedDate picture property as a key, which means that returned values (ids) will be sorted by this property. Second view named by-tag-date will allow us to list pictures with given tag ordered by creation date.

// by-tag-date

//map
function (doc) {
    if (doc.Type == 'picture')
        for (var i in doc.Tags)
            emit([doc.Tags[i], doc.CreatedDate], doc._id);
}

This view emits composite key (an array of values) where first element is the tag, and second is the creation date. This means CouchDB will sort returned values first by tag, then by creation date. By putting constraints on returned key range we’ll be able to select only pictures with given tag. The last view named tags-with-count will allow us to get information of number of pictures assigned a given tag.

// tags-with-count

// map
function (doc) {
    if (doc.Type == 'picture')
        for (var i in doc.Tags)
            emit(doc.Tags[i], 1);
}

// reduce
function (keys, values, rereduce) {
    return sum(values);
}

The map part emits tag as a key and 1 as value. The reduce part takes all the “1s” returned for given tag and sums them. As a result we get picture count for that tag. You can try out those views in Futon.

The PictureController

Last really interesting bit is the PictureController which will cover all of the picture related actions. Its general structure is as follows:

public class PictureController : Controller
{
    public const int PageSize = 6;

    private SessionManager _sessionManager;

    public PictureController(SessionManager sessionManager)
    {
        _sessionManager = sessionManager;
    }

    ...
}

First, actions for adding a new picture.

/// <summary>
/// Displays picture adding form
/// </summary>
public ActionResult Add()
{
    return View();
} 

/// <summary>
/// Saves new picture to db
/// </summary>
/// <param name="picture">The picture desciption</param>
/// <param name="fileUpload">The file upload</param>
[HttpPost]
[ValidateAntiForgeryToken]
public ActionResult Add(Picture picture, string commaTagList, HttpPostedFileBase fileUpload)
{
    try
    {
        if (fileUpload == null)
        {
            ModelState.AddModelError("fileUpload", "Choose a file to upload");
            return View();
        }
        
        if (!fileUpload.ContentType.StartsWith("image/"))
        {
            ModelState.AddModelError("fileUpload", "File has to be an image");
            return View();
        }

        if (!string.IsNullOrWhiteSpace(commaTagList))
        {
            picture.Tags = commaTagList.Split(',')
                .Select(tag => tag.Trim())
                .Where(tag => !string.IsNullOrWhiteSpace(tag))
                .ToList();
        }

        var repo = _sessionManager.GetRepositoryFor<Picture>();
        repo.Save(picture);
        repo.Attach(picture, fileUpload);

        return RedirectToAction("Single", new { id = picture.Id });
    }
    catch(Exception e)
    {
        ModelState.AddModelError("", e);
        return View();
    }
}

User provides picture title, comma separated tag list and selects a file to upload. By using appropriate repository we save the document to the database and then attach a binary file with the picture. Note that saving a document and creating an attachment are two separate operations and they are not included in any transaction. Lack of transactions is a problem with CouchDB as I wrote earlier. In real world application you’d probably want some compensation code, that will remove picture document if creating attachment fails.

Now the action that displays a single picture with a tag list on its side.

public ActionResult Single(string id)
{
    var repo = _sessionManager.GetRepositoryFor<Picture>();
    var picture = repo.TryGet(id);
    if (picture == null || picture.Attachments.Count == 0)
    {
        return new HttpNotFoundResult();
    }

    var model = new SinglePictureModel
    {
        Picture = picture,
        Tags = picture.Tags != null ? LoadTagCounts(picture.Tags.ToArray()) : new TagWithCount[] { }
    };

    return View(model);
}

private TagWithCount[] LoadTagCounts(params string[] tags)
{
    var tagsWithCount = new List<TagWithCount>();
    foreach (var tag in tags)
    {
        var query = _sessionManager.GetQueryFor<TagWithCount>("picture", "tags-with-count", true)
            .All().Exactly(tag);
        var result = query.Execute();

        if(result != null && result.Rows.Length > 0)
            tagsWithCount.Add(new TagWithCount { Tag = tag, Count = int.Parse(result.Rows[0].Value.ToString()) });
    }

    return tagsWithCount.ToArray();
}

We load the picture document using TryGet(id) method od picture repository. Getting a document by its id doesn’t require creating any additional view. Then for all tags associated with this picture we want to load number of pictures under given tag. To do this we query previously created tags-with-count view and constrain returned key range to one tag. We query CouchDB as many times as there are tags, which is not very efficient. There is a feature in CouchDB that allows to load multiple keys with one go to the database. Unfortunately realx-net doesn’t currently support this feature. I sent the library’s author a patch for that, but as of now I didn’t receive any answer.

Next action sends the actual picture file to user’s browser.

/// <summary>
/// Sends picture file to browser
/// </summary>
/// <param name="id">Id of the picture</param>
public ActionResult Show(string id)
{
    var repo = _sessionManager.GetRepositoryFor<Picture>();
    var picture = repo.TryGet(id);
    if (picture == null || picture.Attachments.Count == 0)
    {
        return new HttpNotFoundResult();
    }

    var att = picture.Attachments.First().Value;
    return File(att.LoadBytes(), att.ContentType);
}

In this action we download the image file to the memory and then stream it to the user. There is another way to do this. You could have user’s browser download the file directly from CouchDB by providing it with appropriate URL. But a lot of security issues arise with this approach, so here I decided to download the file to web application’s memory first.

Next action lists all pictures in our gallery.

/// <summary>
/// Lists all pictures
/// </summary>
/// <param name="page">Page number</param>
public ActionResult Index(int? page)
{
    if (page == null)
        page = 1;
    ViewBag.Page = page.Value;

    var pictureQuery = _sessionManager.GetQueryFor<Picture>("picture", "by-date", false)
        .Skip((page.Value - 1) * PageSize)
        .Limit(PageSize)
        .Descending()
        .WithDocuments();
    var result = pictureQuery.Execute();

    ViewBag.HasMore = result.Total > (page.Value * PageSize);

    var pictures = result.Rows.Select(r => r.Entity).ToArray();
    var tags = pictures.Where(p => p.Tags != null).SelectMany(p => p.Tags)
        .Distinct().OrderBy(tag => tag).ToArray();

    var model = new PictureListModel
    {
        Pictures = pictures,
        Tags = LoadTagCounts(tags)

    };

    return View(model);
}

The by-date vie is used and there is paging using Skip() and Limit(). The Total property of query result stores number of all rows in queried view. The value of the view is merely a document id, so we also need to download rest of the document. We could query for each picture separately, which isn’t very efficient. Fortunately there is also option to download whole document related to view entry along with that entry by using WithDocuments().

Next action lists pictures under given tag.

/// <summary>
/// Lists pictures uder given tag
/// </summary>
/// <param name="tag">The tag</param>
/// <param name="page">Page number</param>
public ActionResult Tag(string tag, int? page)
{
    if (string.IsNullOrWhiteSpace(tag))
        return RedirectToAction("Index", new { page = 1 });
    
    if (page == null)
        page = 1;
    ViewBag.Page = page.Value;

    var pictureQuery = _sessionManager.GetQueryFor<Picture>("picture", "by-tag-date", false)
        .Skip((page.Value - 1) * PageSize)
        .Limit(PageSize)
        .Descending()
        .From(new string[] { tag, "Z" }).To(new string[] { tag })
        .WithDocuments();
    var result = pictureQuery.Execute();

    var total = LoadTagCounts(tag)[0].Count;

    ViewBag.HasMore = total > (page.Value * PageSize);

    var pictures = result.Rows.Select(r => r.Entity).ToArray();
    var tags = pictures.Where(p => p.Tags != null).SelectMany(p => p.Tags).Distinct().OrderBy(t => t).ToArray();

    var model = new PictureListModel
    {
        Pictures = pictures,
        Tags = LoadTagCounts(tags)

    };

    return View("Index", model);
}

Two interesting things here. First, we can’t use result.Total in order to employ paging since we don’t want number of all entries in the view, but rather the number of pictures with given tag. For this we use LoadTagCounts() helper method. Second thing is the way we constraint the key range loaded from by-tag-date view. Suppose we have tags like “kittens”, “puppies” and “zebras”. Suppose we load all keys from [“puppies”] to [“puppies”, “Z”].  [“puppies”] is after all keys like [“kittens”, date] and before all keys like [“puppies”, date]. [“puppies”, “Z”] is after all keys like [“puppies”, date] (because no date starts with letter “Z”) and just before entries like [“zebras”, “date”]. This may be hard to grasp at first…

Finally, the action to delete a picture. Only the uploader can delete the picture.

/// <summary>
/// Deletes the picture
/// </summary>
/// <param name="id">Picture id</param>
[HttpPost]
[ValidateAntiForgeryToken]
public ActionResult Delete(string id)
{
    try
    {
        var repo = _sessionManager.GetRepositoryFor<Picture>();
        var picture = repo.TryGet(id);
        if (picture == null)
        {
            return new HttpNotFoundResult();
        }

        if (!picture.CreatedBy.Value.Login.Equals(HttpContext.User.Identity.Name, StringComparison.InvariantCultureIgnoreCase))
        {
            return RedirectToAction("Single", new { id = id });
        }

        repo.Delete(picture);
 
        return RedirectToAction("Index");
    }
    catch
    {
        return RedirectToAction("Single", new { id = id });
    }
}

 

Wrapping it up

There things like views, authentication, routes and setup in Global.asax that I won’t be covering here in the post, but if you’re interested, grab the source.

2 comments:

Erik Tainio Lagusson said...

Hi

I wonder why I can't load the source? It says that it doesn't support this project?

Otherwise a really good post!

Any ideas?

Marcin Budny said...

Hi Erik,

You may need to install ASP.NET MVC 3 for you Visual Studio.

Share