Michael's Blog

Technical things - programming etc

MongoDB LINQ Queries executing on the client side.

For my Final Year Project at University I have been building "Infometrics", something I may well provide more detail on in the future. In this case the only thing you need to know about it that its a C# App written with a MongoDB backend.

Now the crux of the issue I was experiencing was that all database operations seemed to be slow, despite not having that many documents, and indexes set up.  I was getting very confused as to why MongoDB's query profiler was showing an empty query body, and assumed it was something to do with my Expression building code. It all boiled down to the fact that the wrong .Where() method was getting called.


Getting started with Nodejs, MongoDB and Tower.js

I have recently purchased a raspberry pi to play with, my primary reason for getting one was to have a lightweight linux server that I could develop for. There has been lots of good things said about NodeJS recently and so I wanted to see what all of the hype was about and try and create a basic chat application.

Most of my time programming has been spent working on .NET MVC apps and so i wanted to choose a framework where I could reuse my knowledge about MVC. After reading the excellent stack overflow post comparing 2 popular node js frameworks Tower.js and Railway.js.

So after having set up node on my Pi (and compiled mongodb which I ended up not using in the end) this is how I started on my mission to create a basic realtime chat application:


.NET Redis Cache

I fancied a new project so I decided to create a cache provider cabable of storing .NET objects in Redis: 


The class is generic, which means your code does not need to be littered with cast statements, it also utilises the Async capabilities of the booksleeve library on which it is written so you can await cache tasks in your code.

Getting started with logstash Part 2

Getting started with logstash part 1

Statsd & Graphite

In my last post I mentioned statsd and graphite, these are 2 tools that I use in conjunction with logstash and they make a good team!

So the general idea behind running these utilities together is that you can get fancy graphs based off certain events, so for example it might be nice to see a longer term trend of IIS response times for a given website or something to that effect.

I won't go into how to set them up as there are a couple of guides around on how to do it, I can try and point people in the right direction if you have any questions however.

One important-ish thing to note is that if you wish to save a group/single graph (you most certainly will!) you will need to set up a django account when you configure the graphite web app. You can make this work after you have installed it if you decided to skip that step.

Here is how I have configured one of my statsd metrics:


statsd {
    type => "iis"
    tags => ["example.co.uk"]
    exclude_tags => ["_grokparsefailure"]

    increment => "live.example.iis.response.%{response}"
    timing    => ["live.example.iis.responseTime.", "%{timetaken}"]


Events of type "iis" which have the tag "example.co.uk" and did not fail grok parsing are accepted by this output filter. Each time this filter recives an event the metric for that particular response code is incremented (So you could plot a graph of how many files you were missing, 404s, for example). the %{field} notation is used to indicate that this bit of text should be taken from the event itself.

You can have many of these output filters defined for events with different tags, so you can have metrics for different sites etc.

One thing that tripped me up was the storage aggregation configuration, to the lay-man this means that when your storage schema for carbon specifies that metrics should have thier precsion reduced a certain set of rules determine what gets stored as the new value for this time period. However, as the metrics I was tracking did not always have a value (null) I found that for the longer graph periods there was no data. A great post over at http://obfuscurity.com/2012/04/Unhelpful-Graphite-Tip-9 pointed me in the right direction on how to resolve this issue.

Getting started with logstash

Logstash is a tool for doing fancy stuff with log files. Its main purpose is to help administrators/IT staff with the monitoring of logs, specifically it can be used to push logs to a central location where the logs are stored and indexed for later searching. A basic logstash setup (like the one I am implementing at work) consists of log shippers (web servers and database servers) and then one server where the logs are stored, indexed and searched (in my case this is a Ubuntu server).


AINT207 - "Ace Traders" Evaluation

I have now completed the game and acted on the feedback that was given to me when we did the play testing session.

I think that I have produced a well rounded game with a unique concept considering that most of the submissions for the module were sidescrolling flash games built in stencil works. I am glad I chose to implement the game in HTML5 & JavaScript as it is better aligned to my Web Applications Development degree, I have learnt some good practices for general JavaScript development and more details on interacting with the canvas element, in addition to learning how game mechanics should work.

I am happy with my Highscore implementation as it uses an XML datasource and the ASP.NET MVC Web API, my only complaint is that the initial request to start the web application (After the app pool has been unloaded to conserve resources, because of a long wait inbetween requests) can be rather slow, however this will be less of a problem the more popular the game becomes. If I was self hosted (As opposed to shared) I could also extend the life of my app pool to ensure that it was not retired when waiting for requests. The game also has a complete ending which helps make the game feel finished.


AINT207 - "Ace Traders" Feedback

Yesterday we had the feedback/play test session. I recieved some useful feedback from students and lecturers alike, the main points people picked up on were:

  • You cant tell what a station goods a station was buying or selling.
  • It wasnt obvious when you ran out of money/cargo space when purchasing goods.
  • You couldnt check what you had purchased in your hold - and for what price (I dont think I will be implementing price recording, as its part of the challenge to become a good trader!)
  • It was not immediatley obvious (untill your first death) that the pirate base is in the top right and that its bad to go there untill you have the right level of gear.
  • The highscore does not take into account the level of your gear/progress or how much "stock" you have in your hold.
  • It can sometimes be difficult to get started with the trading if the economy is randomised unfairly (E.g Energy is in low demand and or expensive at the power plant).
  • Lack of engaging story.
  • No way of knowing the max quantity of the product.

To address these issues I have: More...

Stack Overflow

Yesterday I posted my first answer to a Stack Overflow question (granted it's not very exciting, but it's still an answer right?!), I have used the site on many occasions before, generally to look up the solution to a problem I have been experiencing, but have not yet posted a question. Stack Overflow is an excellent resource, something I hope to be able to contribute to, I can see how addicting gaining rep and answering questions will be! As exciting as gaining reputation will be I enjoy problem solving, so if I can help other people out as well then that's great!

My "flair":

profile for Spike at Stack Overflow, Q&A for professional and enthusiast programmers

AINT207 - "Ace Traders" Implementation

Rather than just calling my game "Space Simulation Game", a catchy title as that is, I thought it would probably be a good idea to come up with a proper name, especially as I have completed the majority of the functionality. As the game is about trading and "ace" implies you should be good at it, I thought "Ace Traders" would be an apt name for my game. You can find the beta version of the game at spikewade.biz/university/spacegame. I have explained a few of the implementation details below.


Entity Framework 4 - Database Issue

I have been setting up a little web api with MVC4 and entity framework for a university project, everything was running fine on my local development machine, however when I created the database on my host (123-reg), via scripting the database (From SQL Management Studio) and removing commands 123-reg doesnt accept, my application would try and create the database, which my database account doesnt have permission to do. This was very strange behaviour as the database was in a compatible state, DbContext.Database.CompatableWithModel() returned true, but EF still thought that the database didnt exist, DbContext.Database.Exists() returned false. To fix this I found the following post on Stack Overflow http://stackoverflow.com/questions/4558377/entity-framework-4-ctp-5-code-first-development which said that the Database Intialiser needed to be set to null. After applying this change and updating my remote site everything worked as expected.

I am still unsure why EF thought that the database didnt exist, but I am pleased it works now!