Oct 16, 2013

PHP 5.5 and JSON

If you find that after upgrading you computer to a distribution that ships PHP 5.5, be aware that you will need to install the json library and enable it separately

`sudo apt-get install php5-json`
`sudo php5enmod json`

Bear in mind that this is ubuntu, I have not tried with other distributions so I am not sure how this works on those.

Thanks to iteration99 for the heads up

Last time out I introduced a basic idea for an API, this post will be about the basic architecture of the API framework. I have been listening to and reading on API best practices published by APIGEE. Although there are one or two things about this guide that I think pidgeon-hole me into certain standards, I agree in most part with a lot of what this e book has to say, I recommend you give it a browse at some point.

Basic Architecture

Enough of this, on to the main point of this post. At the moment Mokocharlie is built on top of the Zend Framework. I put a lot of effort into that and I am not planning on rewriting it soon, I don't know maybe when we bump up versions to v5 (we are currently in v4). I will be building the API with Laravel 4, although at the time of writing this it has not been officially released, I think there's enough stability in there to actually use it. So without further ado we proceed.

Get Composer

If at this time you do not have composer installed on your computer please go here and install it now. I chose to do a global installation but you can do whatever you like as long as calling composer in your Laravel working directory works.

Get Laravel 4

  • And set up your version control (optional but highly recommended) Get the latest snapshot of the Laravel 4, at this point you can set up a repository to manage your code. We use unfuddle for all our mokocharlie projects and this will be no different. I have however found that the best way to get laravel working as a moving target is track the github repository.
  • I create a branch on my local laravel installation called "laravel-tracker"

    git checkout -b laravel-tracker
  • Add a remote and call it laravel with the url for the laravel git repo

    git remote add laravel git@github.com:laravel/laravel.git
  • Before I start working on anything I make sure I am in master and then do a git pull on the develop branch from laravel

    git checkout laravel-tracker
    git pull laravel develop
  • Do a composer update

    composer update
  • Make sure the application is not broken

    php artisan serve
  • If I am happy with what I see (in most cases I am), I just kill the server and then checkout master and continue working

After a few years of Mokocharlie creating communities around images, it has become necessary to open up this data to partners who want to provide galleries of their own but don't have the time to go out to curate those images themselves.

To this end the most logical path would be to develop an API that leverages the data we already have, I have chosen Laravel for this project. Over the next few weeks I will be blogging about the progress made on this project, from architecture to design decisions etc. The outline will be as follows:

  • Basic Architecture (How clients will connect to this service, authentication protocols, etc)
  • Data Formats/Structures
  • Content provision and distribution
  • Logging and Monitoring
  • Case Studies
  • Further work.

After about an hour of digging around and looking for a way to package a simulation I wrote in Scala for a coleague, I came across this process of doing it. I have reduced it down to a bash script which will be at the bottom of this post but I thought I should go through the steps a bit as the script makes certain assumptions.

The first assumption is that you have the necessary libraries and such to run it they are as follows:

  • Scala Library (That goes without saying if you're going to be writing Scala code) I have a custom path for mine as I want to manually sync my libraries, compilers and so on with the latest versions. I have extracted my copy to ~/Bin/Scala-2.9.1.final this will be the path my bash script will assume, you must change this to your path if it's different.

  • Proguard: I am using version 4.7, version 4.4 comes with ubuntu but, seeing as I like to keep it at the latest and I am impatient my proguard lives in ~/Bin/Proguard4.7

  • Java (This also goes without saying. You will probably need the JDK as well as the JRE so install them both) Proguard uses the java libraries extensively so you need that.

My simulation is in the following directory structure engine | - src | -- | --  Engine.scala | - build-app.sh | - scala-config.pro

The engine->src->Engine.scala is my scala application with the object Engine defined as my entry point this is important for the MANIFEST.MF file the script will be creating

Now the steps

  • First we can create a little folder to hold the output of the scala compiler so we create  a folder called bin inside of the engine folder itself so that the folder structure now looks like this engine | - src | -- | -- Engine.scala | - bin | - build-app.sh | - scala-config.pro by running

    mkdir bin

then run the following while you're inside engine which from now on I will refer to as root

scalac -sourcepath src -d bin src/Engine.scala

this will compile the Engine.scala code into java classes inside of the bin folder

  • This completes the scala bit, everything else following is run with Java so we need a MANIFEST.MF file to compile a jar

    echo "Main-Class: Engine" > MANIFEST.MF

this will do it for you. Bear in mind you are still at the root level

Now at this point we change directory to the bin folder where scalac has created our classes and compile our jar using the manifest we have just created so this should do it:

cd bin
jar -cfm ../engine-no-scala.jar ../MANIFEST.MF *.*

You can obviously do this a different way by staying in the root and making sure your path is correct, personally I found this easier.

This above code will create our engine called engine-no-scala.jar with the MANIFEST.MF file bear in mind at this point if you ran java -jar engine-no-scala.jar it may fail if your scala code required the scala library to run mine does so now I need to package this WITH the scala library into a complete self-contained java application now earlier I did say that my scala home was in a custom location so we need to keep that in mind for the next section


  • Not only does proguard include the required libraries and so on, it also obfuscates and discards any unused classes and packages this is very nifty for distribution as it cuts down by orders of magnitude the size of your application.When I did this without proguard my application was over 8 MB because the entire scala library was in my jar after I finished with proguard I am looking at an 8 KB application, and I think I can even get it smaller if I optimise my code a little bit morebut that's for another time. Proguard however needs a configuration file to work properly in my case, I went to their site and downloaded a sample scala-config file (click on the Scala link) I just made the following changes to the vanilla one they provided

    -injars engine-no-scala.jar -injars /home/kwabena/Bin/scala-2.9.1.final/lib/scala-library.jar -outjars engine.jar -libraryjars <java.home>/lib/rt.jar

    -dontwarn scala.** -verbose

I just changed the path to my jar file and my scala library and I also turned on verbosity (I like to know what's going on) my build script now has the following lines

cd ../
echo "Adding Scala library, obfuscating..."
proguard.sh @scala-config.pro

after that you can remove any side effects from the script with

rm -rf bin MANIFEST.MF engine-no-scala.jar

bear in mind that we changed the our current workind directory to the root level after a few minutes of compiling, obfuscating, optimisation and shrinking you should have a nice jar to run by itself on a computer with ONLY the JRE installed

The complete script follows you can just copy this, save it into a file called "build-app.sh" as it is in my folder structure change the permissions to execute with

chmod +x build-app.sh

and run it


echo "Creating required directories"
mkdir bin

echo "Compiling scala sources"
scalac -sourcepath src -d bin src/Engine.scala

echo "Creating Manifest"
#build manifest
echo "Main-Class: Engine" > MANIFEST.MF

echo "Building java archive from scala classes"
#create jar
cd bin
jar -cfm ../engine-no-scala.jar ../MANIFEST.MF *.*

cd ../

echo "Adding Scala library, obfuscating..."
proguard.sh @scala-config.pro

#remove fluff
echo "Removing fluff..."
rm -rf bin MANIFEST.MF engine-no-scala.jar

I hope you find it as useful as I would have should this have been available earlier.

Oct 11, 2011

Web frameworks for Scala

The best way for me to learn a language generally is to build a project using that language. I have decided to start on this Scala journey and so far it's been all theory and very little practice. In any case I have always been a web developer and that is where I would like to stay. So a web project in Scala is naturally the path I would want to go with. I kept stumbling across the Lift framework, I like Lift but it's a bit tricky for me right now at this level and so I will perhaps visit it later.

I then came across the parsimonious Scalatra. I love it it's light elegant and doesn't require you to do too much to get started. I have not used it long enough to say that this is going to be my weapon of choice for any future developments but it's a very strong candidate.

I only just now discovered the Play framework and on first glance it's a brilliant looking MVC framework, honing in on my Zend experience with PHP. It feels like Zend, I have not actually started writing anything in it yet but I am going to attempt a full blown web project in it and see how that goes. Perhaps something for an EC2 instance.

Sep 29, 2011

Setting Up SBT on Ubuntu

So I have had to re-install my OS several times over the past month mainly due to my impatience (installing 11.10 beta and not having it working properly) and I have found that my SBT configurations generally evaporate with every re-installation so I wrote a little script to automate the whole process. A bit like my previous get Scala script so once again just copy and paste this into a text file, make it executable with chmod +x and run it


cd ~

echo --------------------------------------------------------------------------------------------
echo - Getting SBT Launcher
echo --------------------------------------------------------------------------------------------
wget http://typesafe.artifactoryonline.com/typesafe/ivy-releases/org.scala-tools.sbt/sbt-launch/0.11.0/sbt-launch.jar

printf 'java -Xmx512M -jar `dirname $0`/sbt-launch.jar "$@"' > sbt

chmod +x ./sbt

echo --------------------------------------------
echo - Move sbt and sbt-lauch to share
echo --------------------------------------------

sudo mv sbt /usr/share/sbt
sudo mv sbt /usr/share/sbt-launch.jar

echo --------------------------------------------------------------------------------------------
echo - symlinking sbt and sbt-lauch to /usr/bin/{sbt, sbt-lanch.jar}
echo --------------------------------------------------------------------------------------------
sudo ln -s /usr/share/sbt /usr/bin/sbt
sudo ln -s /usr/share/sbt-launch.jar /usr/bin/sbt-launch.jar

I suppose, when I get time I can make this accept parameters so you can specify the version of SBT you want and the file name you want to use etc but for now this should suffice... comments welcome.

Having followed instructions from here, which I thought was really easy, I put together a little bash script to do exactly that. And being a sucker for the latest I went with the installation. I am honestly not sure why Ubuntu still has 2.7.7 in its repository. Ubuntu 11.10 (oneiric)  now has scala 2.9.x so you may want to use that instead of this script but if you want to keep up to date with the latest Scala release then:

Simply copy this and paste into a file, name it whatever you want, and make it executable with chmod +x


cd ~

echo --------------------------------------------------------------------------------------------
echo - Getting Scala
echo --------------------------------------------------------------------------------------------
wget http://www.scala-lang.org/downloads/distrib/files/scala-

echo --------------------------------------------------------------------------------------------
echo - Decompressing
echo --------------------------------------------------------------------------------------------
tar zxf scala-

echo --------------------------------------------------------------------------------------------
echo - moving scala to /usr/share/scala
echo --------------------------------------------------------------------------------------------
sudo mv scala- /usr/share/scala

echo --------------------------------------------------------------------------------------------
echo - Symlinking executables
echo --------------------------------------------------------------------------------------------
sudo ln -s /usr/share/scala/bin/scala /usr/bin/scala
sudo ln -s /usr/share/scala/bin/scalac /usr/bin/scalac
sudo ln -s /usr/share/scala/bin/fsc /usr/bin/fsc
sudo ln -s /usr/share/scala/bin/sbaz /usr/bin/sbaz
sudo ln -s /usr/share/scala/bin/sbaz-setup /usr/bin/sbaz-setup
sudo ln -s /usr/share/scala/bin/scaladoc /usr/bin/scaladoc
sudo ln -s /usr/share/scala/bin/scalap /usr/bin/scalap

So after about an hour of research and experimenting, I found out how to make Scala work with MySQL without too much hassle, I found a really helpful guide here. Going down the SBT route, I first created a project and all that, then inside of the build folder I created a Scala class that ensured that when I started building, I would have the right dependencies downloaded and ready.

import sbt._

class MyDataEngineProject(info: ProjectInfo)
    extends DefaultProject(info) {
  // Declare MySQL connector Dependency
  val mysql = "mysql" % "mysql-connector-java" % "5.1.17"

Once you have got this in project/build/MyDataEngineProject.scala, you are ready to start the connection stuff. In src/main/scala you can start creating your application here. I use the "def main" approach so when i start from the command line i simply pass in my parameters and I'm ready to go:

import java.sql.{Connection, DriverManager, SQLException, ResultSet}

object DataEngine {

    private var driverLoaded = false

    private def loadDriver()  {
            driverLoaded = true
            case e: Exception  => {
                println("ERROR: Driver not available: " + e.getMessage)
                throw e

    def main(args: Array[String]) {
        if (args.length < 1){
            println("No arguments provided, exitting...")
        println ("Attempting to load MySQL JBDC Driver...")
        this.synchronized {
          if(! driverLoaded) loadDriver()

        val conString = "jdbc:mysql://localhost:3306/db_name?user="+args(0)+"&password="+args(1)
        val con = DriverManager.getConnection(conString)
            val handle = con.createStatement(ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY)
            val rs = handle.executeQuery("select * from auth limit 1")
            println ("Connection established...")
                println("User Handle: "+rs.getString("handle"))
            case e: Exception  => {
                println("ERROR: No connection: " + e.getMessage)
                throw e
            case _ => println("A problem!")
            println("Closing connection and exiting...")

Save the file as Main.scala and you can just type in run with your db username and password as space-separated arguments and you're ready to go. Of course you will need to change some bits in here to reflect your database schema, but aside that it's that simple. In my next post I will work on inserting data and updating data as well.

Jul 26, 2011

Moving to Scala

So after about 8 - 9 years of PHP and enjoying it thoroughly, I have decided to default to Scala pronounced {skah-lah}. I am not going to go into any reasons why I am moving but all I can say is that, I think the Lift Framework is amazing and Scala is a lot more elegant in my opinion, very expressive and perhaps my biggest reason is that I would like to expand those horizons, (I was getting a bit too comfortable in this space).

Apr 8, 2011

Using images in LaTeX

Note to self and perhaps others who might be wondering why your images are not displaying in your tex document.

Despite using the correct syntax:


    \caption{Fig 1: image}

` You may find a box with with only the image path being displayed as your image. Please check that you are not using the "draft" argument at the top of your document when specifying your document class. A bit like


Make sure there is no "draft" in there so that it reads


Spent about half an hour trying to figure out why everyone else's images were displaying except mine.

Return button