Architecture

.Net Developer's Guide to Getting Started with Cassandra

Eden Ridgway — Thu, 05 Nov 2009 22:32:51 GMT

I've been quite curious about the various BigTable like implementations as they could have some bearing on what I am involved with at work. After reading various write ups about the competing approaches (e.g. HyperTable, HBase, Dynomite, Voldemort, Mongodb) I decided that Cassandra looked like the best option for what I'm after. What I didn't realise was that it wasn't going to be as straight forward as it appears to set it all up as there is no end-to-end guide of getting started with it. So the objective of this post is not to explain Cassandra but save you several hours of jumping all over the web trying to work out what you should do next and how to generate the necessary C# Cassandra Thrift client code. If parts of the post don't apply, e.g. VM setup, simply skip ahead to the parts that do. If you want more information about Cassandra you should check out the following posts:

So to get Cassandra up and running in a Virtual Machine (or physical if you choose) you should follow the following steps (valid as at 5 Nov 2010):

Create a virtual machine to host Ubuntu 9.10 x86 Desktop. I use VirtualBox as my VM host on windows so I will refer to this here but feel free to substitute it with you favourite VM software. The VM I created has 384MB allocated to it with an 8GB virtual hard drive using a bridged network adapter (so it can be accessed from the Windows XP host). Once you've gone through the install process install the client tools so that you change the screen resolution, share folders, etc. To do this with virtual box go to Devices > Install Guest Additions and then open up a terminal window and execute

sudo /media/cdrom/VBoxLinuxAdditions-x86.run

Once this is complete you will need to reboot your VM.
Now it is time to install Cassandra/Thrift prerequisites and software that we will to set everything up:

sudo apt-get install libboost-dev automake libtool flex bison pkg-config g++ -y
sudo apt-get install sun-java6-jdk -y
sudo apt-get apt-get install ant -y
Once that is done we can install other useful Ubuntu software that we will use:

sudo apt-get install rapidsvn -y
sudo apt-get install monodevelop -y

I also like to install the following:

sudo apt-get install nautilus-open-terminal -y
sudo apt-get install samba smbfs smbclient winbind -y
sudo apt-get install nautilus-share -y
You may discover that there is an issue with your Java runtime home directory. If that occurs, select Sun's Java Runtime as the default Java version by running the following command from the terminal window and selecting the third option:

sudo update-alternatives --config java
Now open up Nautilus by going to Places > Home Folder. Right click and create a Development folder. Within the Development folder create another two folders: Cassandra and Thrift.
We can now get the source for Cassandra and Thrift. Run RapidSVN by going to Applications > Programming > RapidSVN. Check out the source from SVN by going to the RapidSVN menu option Repository > Checkout. In the URL enter http://svn.apache.org/repos/asf/incubator/thrift/trunk and for the Destination Directory browse to the Thrift development folder. So it should look like this:

Now get the Cassandra code by doing the same thing but entering http://svn.apache.org/repos/asf/incubator/cassandra/trunk as the URL and selecting the Cassandra directory.
Once we have all the code we are ready to compile Cassandra, so let's do that by executing the following commands from the terminal window:

cd ~/Development/Cassandra
ant
Start the Cassandra server by opening a new terminal window and running the following commands:

sudo ~/Development/Cassandra/bin/cassandra -f

The running server will look like the screenshot below. You should take note of the thrift binding address of localhost/127.0.0.1:9160.
Now we want to get communicating with the server, so we will need to compile Thrift so that we can use it to generate our C# client:

cd ~/Development/Thrift
./bootstrap.sh
./configure
make
sudo make install
Now we can finally generate a C# client for the Thrift calls to Cassandra:

cd ~/Development/Cassandra/interface/
~/Development/Thrift/compiler/cpp/thrift -gen csharp cassandra.thrift

This will create the following directories inside of the Cassandra/interface folder: gen-csharp/Apache/Cassandra. These files along with the Thrift.dll will be copied into a new demo project we will create using MonoDevelop in the Development folder. So again in the terminal window execute:

mkdir ~/Development/CassandraDemo
cp ~/Development/Cassandra/interface/gen-csharp ~/Development/CassandraDemo -R
cp ~/Development/Thrift/lib/csharp/Thrift.dll ~/Development/CassandraDemo
Open up MonoDevelop and create a new console application by going to File > New > Solution. Now do the following:
1. Select Console project
2. Enter CassandraDemo as the name
3. For the location browse to the Development folder that we created in your home directory.
4. Uncheck the 'Create separate Solution directory' check box.
5. Click the Forward button
6. Don't make any changes on the next screen and just click Ok.
7. Right click on the CassandraDemo project in the left tree view and select Add > Add Files.... Navigate to ~/Development/CassandraDemo/gen-csharp/Apache/Cassandra and select all the files in the folder. You will now have a project that looks like this:
8. Add a reference to the Thrift assembly by right clicking on References and selecting Edit References. Change to the .Net Assembly tab and double click on the Thrift.dll and click OK.
9. Now we can add the sample code to test it all out. So double click on Main.cs to ensure that is the class in focus in the editor. Replace the contents of the file with this:
  
  namespace CassandraDemo
  {
      using System;
      using System.Collections.Generic;
      using System.Diagnostics;
  
      using Apache.Cassandra;
      using Thrift.Protocol;
      using Thrift.Transport;
  
      class Program
      {
          static void Main(string[] args)
          {
              TTransport transport = new TSocket("localhost", 9160);
              TProtocol protocol = new TBinaryProtocol(transport);
              Cassandra.Client client = new Cassandra.Client(protocol);
  
              Console.WriteLine("Opening connection");
              transport.Open();
  
              System.Text.Encoding utf8Encoding = System.Text.Encoding.UTF8;
  
              long timeStamp = DateTime.Now.Millisecond;
              ColumnPath nameColumnPath = new ColumnPath()
                                          {
                                              Column_family = "Standard1",
                                              Column = utf8Encoding.GetBytes("name")
                                          };
  
              Console.WriteLine("Inserting name columns");
  
              //Insert the data into the column 'name'
              client.insert("Keyspace1",
                            "1",
                            nameColumnPath,
                            utf8Encoding.GetBytes("Joe Bloggs"),
                            timeStamp,
                            ConsistencyLevel.ONE);
  
              client.insert("Keyspace1",
                            "2",
                            nameColumnPath,
                            utf8Encoding.GetBytes("Joe Soap"),
                            timeStamp,
                            ConsistencyLevel.ONE);
  
              //Simple single value get using the key to get column 'name'
              ColumnOrSuperColumn returnedColumn = client.get("Keyspace1", "1", nameColumnPath, ConsistencyLevel.ONE);
              Console.WriteLine("Column Data in Keyspace1/Standard1: name: {0}, value: {1}",
                                utf8Encoding.GetString(returnedColumn.Column.Name),
                                utf8Encoding.GetString(returnedColumn.Column.Value));
  
              Console.WriteLine("Getting splice range");
  
              //Read an entire row
              SlicePredicate predicate = new SlicePredicate()
                                         {
                                            Slice_range = new SliceRange()
                                                          {
                                                              //Start and Finish cannot be null
                                                              Start = new byte[0],
                                                              Finish = new byte[0],
                                                              Count = 10,
                                                              Reversed = false
                                                          }
                                         };
  
              ColumnParent parent = new ColumnParent() { Column_family = "Standard1" };
              Dictionary<string , List<ColumnOrSuperColumn>> results = client.multiget_slice("Keyspace1",
                                                                        new List<string>() { "1", "2"},
                                                                        parent,
                                                                        predicate,
                                                                        ConsistencyLevel.ONE);
  
              foreach (KeyValuePair<string, List<ColumnOrSuperColumn>> resultPair in results)
              {
                  Console.WriteLine("Key: {0}", resultPair.Key);
  
                  foreach (ColumnOrSuperColumn resultColumn in resultPair.Value)
                  {
                      Column column = resultColumn.Column;
                      Console.WriteLine("name: {0}, value: {1}", utf8Encoding.GetString(column.Name), utf8Encoding.GetString(column.Value));
                  }
              }
  
              Console.WriteLine("Closing connection");
              transport.Close();
          }
      }
  }
10. Hit F5 and watch the Application Output window. This is what you should see:

Design By Contract Using XC#

Sat, 24 Sep 2005 00:35:18 GMT

Design by contract is an approach to programming where one establishes pre and post conditions when calling methods or modules. The idea is that by asserting that by defining and validating the interface in this manner bugs are reduced and are easier to find.

I found a framework for doing this called eXtensible C# (XC#) which allows for pre and post conditions using attributes. They use a compiler of sorts to do compilation after the C# compiler for you code has been invoked from VS.Net. Here is an example of how one would use XC#:

[Requires ("app != null")]
void UpdateApplication (Application app)

XC# comes as an add-in to VS.Net that appears to hook into the post build event. You can also add obfuscation to your project by simply adding an Obfuscate attribute to the assembly.

Some of the alternatives to this approach are:

Using Aspect Oriented Programming (AOP) to automatically intercept the calls. One should be aware that this is most likely quite slow. Here are some example open source AOP frameworks for .Net:
Manually coding the rules at the beginning or the end of each method call. There is a 2002 CodeProject Design by Contract article about doing this.
Using a home brewed attribute validation system [which is really what XC# is doing in any case]

CSLA.Net

Fri, 08 Jul 2005 01:59:31 GMT

I bought Rockford Lhotka's Expert Business Objects book a while ago. In my current project, I'm considering using a domain design model that may include CSLA.Net elements, so I thought I'd quickly bookmark some useful resources:

Dave Hayden has a chapter by chapter summary of the book
Rockford's web site
A Got Dot Net CodeSmith Template collection
ActiveObjects for CSLA.Net - handles things like custom windows forms databinding

I also read an interesting blog posting by Paul Wilson, where he said the following about CSLA.Net:

"There are many styles and possible solutions in .NET -- I have used CSLA on a major project and do not want to do so again -- but I can say the same about many other techniques I have used in the past too...

...I would say it didn't work -- although it was a heck of a lot of work in my opinion (still ongoing when I left). Part of that work was in removing a lot of the features that we simply did not need and which were causing excessive memory use. Granted our system was not typical (lots of large recordsets and on Citrix), and I blame management for not listening better there. But the real "work" to me was in creating my business objects, and making sure that all the dependencies were correct also. Some of that can certainly be alleviated in many projects with code gen, but I simply believe there are simpler ways to do things...

So I want something simple for my teams to use and learn and maintain, while still being flexible and easy to extend. I'm simply at the place in my career where I'm tired of the same old things -- and I strongly believe I've found better ways."

For me, there are two aspects that I'm interested in taking from CSLA.Net, the business rules approach and the collection of errors. I am prone to changing my mind so don't hold me to that. I want to have a framework that is obvious and simple but where the logic lives in the business logic layer as opposed to the UI. Our current Active Record style design pattern however tends to promote the coding of logic in the wrong place and this is a product that has a fair amount of logic that cannot reside in the UI. I get accused of making things too complicated, but what I want us to create reusable (where it makes sense) and structured solutions. Sometimes that involves a little more complexity than a Transaction Script type design pattern.

I try to keep things as simple as is possible while still achieving reusability and modularity. I'm only too happy to get someone over to my desk to take a look at the structure and tell me if they find it understandable. If you call copying and pasting the same errors err... I mean code all over the application simple, you will pay a maintenance tax later. It's that simple. Also it appears that some people are unable to grasp relatively simple concepts like inheritance. As soon as you use inheritance you get told you are making the solution too complicated even though you've reduced the delivery time and maintenance complexity (to some) dramatically. Come on, if all the classes inherit from a base class and you can't find the logic, have you thought of looking in the base class? That big comment telling you that the base class collects shared information about the process surely must be a big give away.