seawolfsanctuary

RVM + rSpec = Compatibility Fun!

Mon, 26 Sep 2011 19:00:06 +0100

Rubyists make a decision which version of which flavour is their preferred, either generally or for that project. But, does it not seem that sitting inside their chosen one a little closed-minded and unfriendly to others’ preferences? It does to me, so I’m combining the magic of RVM with my rSpec testing to make my software cross-version — and even cross-Ruby — compatible.

Erm… Ahem.

Before we go any further I need to check that you took my advice and integrated RVM with your Ruby projects, yeah? Good, I hope it’s been a good experience for you to chop up your development environment into per-project workspaces; if you haven’t, GTFO.

Great: you guys and girls left are the ones I want to deal with. Now you must prepare for your next challenge: combining RVM with that ever-so-awesome testing tool rSpec. Don’t worry if this seems over-the-top for your project; it’s worthy practice in even the most modest of code-bases. I’ve only the smallest of personal projects and only one I deem worthy of such a practice. Coming to the conclusion that I’d like my rubident code to run in more than ten minutes time, I have to make sure it’s going to work the second, third and hundredth time round. How? Test it!

The Project

Embarking on rubident out-of-the-blue meant I hadn’t looked into how a Ruby project could consume the web services and APIs it does so I just went ahead and wrote some code. Yep, I got myself into development-driven testing; I wanted to find my feet in how best to organise it before fighting the dreaded red-coloured output since I didn’t know rSpec at the time either! Around about here rubident looked like this:

OOP, not just a script as it once was (turns out doing it the proper way is just as simple!)
set multi-service support
any number of accounts on those services
oAuth (v1) authentication through them
reading and writing pretty much all the expected streams; home, replies, direct messages etc.
a couple of helpers to sort out the dates
hosting on Gitorious and GitHub.

Since the specifications for the project is to eventually support everything the API does (what more can a guy do?) the one thing missing from that list is the big one: tests. I had to get testing up there, so now that I have something to work with I can learn how Rubyists test!

Testing: And So It Begins

At the time of writing, the documentation for the newer rSpec (v2) lacks greatly from that of it’s predecessor, which is quite unsettling for a beginner. This made it quite difficult to pick up rSpec as I didn’t want to read documentation and tutorials for v1 and find out I’d need to re-write my attempts. Admitting defeat, I just wrote some ruddy code. I already had a list of features that rubident did:

start a client instance
make sure we load some gems (JSON et. al.)
make sure we load some files (helpers)
support a number of (technologically very similar) services
add accounts on those service
understand oAuth (ahem, I’m doing this one last…) to add those accounts
read various timelines
parse and order various stuff out of each stream
post through an account

Writing this list made writing the list of tests easy:

load ‘rubident.rb’

When starting up:

it should create a client to work with
it should load at least some of the required gems

When setting up services and accounts:

it should read a file of supported services
it should read a file of set-up accounts

I’m sure you can see where I’m going with this; adding this list of requirements as the rSpec file for my main rubident class — spec/rubident_spec.rb — gave me an instant template for testing my code. I’m not going to go into the actual code, hat’s up to you to make sure your tests actually do, but I will say that:

it should create a client to work with
it “should create a client to work with” do
@client = Rubident.new
@client.should_not be nil
end

The Awesome

Here’s the TL;DR of this post if you’re using RVM, rSpec and have written some code and tests. I’ve plumped for a plain ol’ Bash script to do the following:

list all the Rubies installed on my system through RVM, in a machine-readable format (the strings bit)
switch to the rubident environment for each installed Ruby and version
ensure all my Gems are installed as per the Gemfile
run my rSpec tests for this particular Ruby and version, coloured and listing my tests
carry on to the next Ruby or version
at the end, switch back to my preferred Ruby and version.

for RUBY in `rvm list strings` ; do
rvm use $RUBY@rubident && bundle install && rspec -cfn spec/*.rb ; echo
done

rvm use 1.9.2@rubident

This gives me 100% compatibility for all my installed Rubies as far as I have tested, in one command. By having older Rubies installed, I can maintain backward-compatibility too. Of course, this is limited to as much backward-compatibility as the gems allow for, and how far my tests go but I’ve found this is enough to inspire my confidence that I am writing well-working code

I immediately saw a failure resulting from changes between versions: MRI 1.8.7 can require both gems and files and load files, whereas v1.9.2 only require gems but still load files. By getting them green without version-specific code, one can ensure it is running at a sort of baseline, without specific features. You can also include optimisations for specific Rubies in your code and have them tested, since we are switching between whatever is installed — but that’s up to how you want to work.

Thanks for reading! I hope you find my post useful if not interesting, and appreciate any (appropriate!) comments.

By the way, I use the term Rubies to group not just all Ruby implementations but also every version of them. I thought it might be useful to clarify this. How do most use the term?

fsfe UK Meet #1

Mon, 25 Apr 2011 12:14:29 +0100

fsfe UK Meet #1:

The evening of Thursday 7th April saw the inaugural meeting of fellows in the Free Software Foundation Europe’s British branch. Generously hosted by MaDLab, Manchester’s…

RVM: Ruby Projects in a World of their Own

Sat, 19 Mar 2011 19:30:00 +0000

Whether you’re just starting to use Ruby or have been for a while, when you pull in the first Gem you had better be using RVM, the Ruby Version Manager. Why? Well, RVM allows you to easily set-up not only multiple versions of Ruby but also per-project sets of Gems, and for you to seamlessly switch between them. By having a number of persistent Rubies on your system simultaneously, each project lives safely in it’s own world so you can reliably separate and distribute them without leaving anything out. All this can be done away from your system’s own Ruby version and Gem set, too.

Approach

The common approach is to set up the Rubies inside your own Home directory, under the .rvm directory, but I’ve had no problems with a system-wide installation. (Except, that is, for remembering that my Rubies are installed under /usr/local and not in my Home directory, so needing elevated privileges to use RVM or some user-group work-around.) A quick round-up of the commands and methods for setting up and using system-wide RVM is the purpose of this post.

Bear in mind that Ruby is cross-platform and so all the tools around it have to be too. That’s not to say there are problems with it, just something to keep in mind that any stumbling block may be a Mac OS / Linux / Windows thing rather than the tool you’re having problems with.

Installation

We begin by, you guessed it, installing RVM. The easy way is by its script, which is downloaded and ran:

curl -k https://rvm.beginrescueend.com/install/rvm > ~/rvm_install.sh
chmod +x ~/rvm_install.sh
./~/rvm_install.sh

I’m using SliTaz Linux, whose user management tools are slightly different from those in a number of other distributions because of the glorious (yes, sarcasm) BusyBox. Because of this, the RVM user group automation doesn’t work, so I need to run:

sudo addgroup seawolf rvm

to add my user to the rvm group.

Like all great software, RVM’s user-level configuration is found in one file. Let’s take a look:

cat /usr/local/rvm/examples/rvmrc

I’m not going to go over this configuration as the defaults should be more than fine, and you can read a lot more than I know at the RVM website.

Speaking of documentation, you should really take a look at your platform-specific bugs and notices, either from the website or with the command:

rvm notes

I had a few gotchas in learning RVM but nothing huge; there’s plenty around the web and I’ve heard the IRC channel is particularly excellent for support (and chat, Wayne’s a nice guy).

Using RVM

With the paperwork out of the way, it’s the moment you’ve been waiting for: installing Ruby! You are in fact coming to download, compile and install a Ruby with one, yes one, command. This one, to be precise:

rvm install 1.8.7

Whether or not this actually works is a matter for your system, but that’s how it done. I’m not going to write the standard software compilation blurb, so any problems: you know what to do! I like to completely overwrite a failed compile/install with it’s original source before attempting it again, so this time instead run:

rvm install 1.8.7 --force

Lovely. Now, the fun bit: let’s do a similar thing with another Ruby, say the latest Ruby Enterprise Edition:

rvm install ree

Whoo! You’ve now your system’s own Ruby (whatever that may be), 1.8.7 and even REE installed and ready to go, all in one system! But how can you actually use them? Simple:

rvm list

You should see proof of multiple Rubies. Pick any one of them, for example:

rvm use 1.8.7

Note there’s one unlisted; ‘system’ is whatever Ruby your system has installed, usually by your package manager.

RVM & RubyGems

So, now that you know how to switch between them, you can install some Gems for a particular Ruby. Gems are as usual installed under Ruby’s path, this doesn’t change with RVM. By switching with ‘use’ you also switch to the Ruby’s Gems. Let’s have a look to see which Gems are installed with your ‘used’ Ruby. What would you normally do? Probably this:

gem list

RVM takes control over Gems since it’s all part of one particular Ruby by keeping everything under the .rvm directory inside your home. Every Ruby, version and gem lives in there. Have a go at installing some Gem you like (say, Bundler) with:

gem install bundler

To illustrate the Gem handling, let’s switch back to the system Ruby with:

rvm use system

Now, if you ask which Gems are available now, you should get back your original list:

gem list

You should see your previous gems listed there and not whatever one you just installed. Switch back and watch it change again!

Using RVM with Multiple Projects

Long-time or occasional developers may have projects written using various versions of Ruby, to the point that each project is different. There is a handy way to remember which should be used, and that is to let RVM automagically switch between them for you! It will do this when you change into the project’s root directory by noticing a ‘.rvmrc’ file, with the contents as:

rvm <version>@<project-name> --create

You may see what this file is really doing: these instructions are carried out by RVM each time we change into a directory and the file is detected, so it is automatically switching to a project-centric gemset each time it loads this file. I imagine other RVM commands can be used here too, should your workflow deem it necessary. It’s great practice to use this file as soon as you create a new project, so it’s not mixed-up with anything else, as in a similar workflow to Bundler. The two happen to work very well together, as you’d suspect.

Thanks Wayne!

So, there we have multiple versions of Ruby existing on the same machine at the system level, and numerous projects switching to their appropriate Ruby and Gems. RVM is a fantastic tool for anyone with multiple Ruby projects, or even just one since it allows for testing over multiple Rubies very succinctly. A big, big thank-you to Wayne, the creator of RVM for his excellent and hard work!

On Firewood And Kindling

Sat, 27 Nov 2010 22:55:00 +0000

Thursday saw the latest monthly meet of Linux users in and around Chester; its Liverpudlian counterpart to follow this Wednesday. A member of each has recently added to their gadgetry a popular proprietary electronic book reader, the Kindle from onile retailer Amazon.

The thin, “revolutionary wireless reading device” aims to bring books into the digital age, both at leisure and on the move. A user purchases texts from Amazon’s online store or transfers DRM-free PDF documents to the device to read on it’s non-glare, paper-simulate screen. The wireless networking and 3G mobile Internet features are both free and world-wide, attractive to techies and non-techies alike to provide new and previously purchased material instantly, along with full Internet access on the go. Amazon even foot the 3G bill in the hope of re-cooperating enough from store purchases. All in all, the Kindle - and other eBook readers I dare say - is rather tempting to commuters and those wanting to travel light, taking all the good bits of a good book and rounding off the inconveniences of a cumbersome paper companion with modern technology.

Sounds too good to be true? Possibly. I’ve recently reopened James Fray’s a million little pieces, and soon to start its sequel, my friend leonard. Both are available to buy as eBooks but for double what I paid for my second-hand paperbacks. Also on my reading list are crime fiction novels from Christopher Brookmyre and Jonathan Kellerman, only a selection of which are available electronically. Somewhat expected given they are hardly world-renowned classics! As are most of my personal PDFs, technical references that quickly become irrelevant as technology progresses. To compound this, any future downloads would be in the proprietary AWZ format, which may restrict me in the future should Amazon decide to lock-down. On the plus side, they can be read on other devices like my netbook and my Android mobile phone through Amazon’s Kindle application (which is sadly not available for Linux).

So: the perfect travelling companion if it can give you what you want. Until the issues are resolved, I’ll stand by my trusty netbook for a good balance of power and portability, alongside my paperbacks for availability and financial realism.

The turning point? Perhaps it is one already on the horizon: daily newspapers. When done with my book, I regularly read the… easy on the mind Metro and regional daily Manchester Evening News during my lunch-break or on the train home from work. If these were delivered to a Kindle everyday for a negligible amount alongside my livre du jour, I would become again interested. The Financial Times, Independent and others are already published in this way through a subscription, but I personally doubt my selection are considering such a distribution given they are already free and widely available. To those I say: prove me wrong.

Ruby/GTK+ Development in SliTaz: Part II

Fri, 03 Sep 2010 11:00:00 +0100

Last time, we got off to a flying start with a simple desktop application powered by Ruby. We shall run through completing the wizard with a simple selector widget (a spinner) and displaying that result in the final page. This is just to use one of the many widgets, show off the wizard and how to do some magic in it.

Before we get back into Glade³, you may notice a small ‘feature’ in the wizard: it’s fixed at the introductory page! This really a feature; as a wizard, we must flag each page as complete before being allowed to move on. But how can we do this?

Enter, Ruby!

You may have noticed Gladex created second file, the “callbacks” file no less. This is where the UI and code come together; this is where we can work out if the page is complete. At first this file is more or less empty, the only code here is a bizarre-looking constructor with a puts statement inside:

def init_callbacks(xmltree)
    $tree = xmltree
    puts "initialize"
end

The clue this gives us is indeed the puts statement because, when the program is run from a terminal, this initialize can be seen. In fact, the main class file uses this line:

init_callbacks(@glade)

We need to add callbacks from the Glade³ file to Ruby code. This is where it all ties together! Each event that may occur can be associated with a method. Each mouse click, movement, selection or any other event can trigger a method in the callbacks file.

Before You Start, Know How To Stop

One interesting feature you may notice is that we cannot yet exit the wizard without killing it from the terminal (Ctrl+C). Okay, that’s not a feature at all, it’s just darned annoying. In the metaphorical bug-tracker, that’s probably MUSTFIX. To do so, we should attach a callback to the Apply button; similar to before:

Ensure the main window is selected in Glade³.
In the Signals tab of Properties area, open GtkAssistant and select apply.
Create a name for the method as the Handler, I have chosen quit_app.
Back in the callbacks file, create this method. There’s not much to it as we can just quit straight-away, for now:

def quit_app()
    Gtk.main_quit
end

That should be fairly readable; all that is needed to note is that it comes directly from the Gtk module, and that doesn’t really matter.

Go, Go, Go!

The above task shows how the Glade³ file links with Ruby code; it is fairly evident that Ruby/Gtk programs truly are powered by Ruby. Now we can go on to do more with the program. The first thing we should do is sate when we can proceed from a page. This is done by ‘flagging’ the page as complete and allowing the Next button to become active, for now it will be as we load each page:

Ensure the main window is selected.
In the Signals tab of Properties area, open GtkWidget and select configure-event.
Create a name for the method as the Handler, I have chosen prepare_page as we shall use it with every page.
Repeat the previous step for the GtkAssistant > prepare signal.
Back in the callbacks file, create this method. As we will use it on every page, it can be quite generic, without much logic:

def prepare_page()
    curr = @glade["assistant1"].get_nth_page(@glade["assistant1"].current_page)
    @glade["assistant1"].set_page_complete(curr, true)
end

Hopefully that isn’t too complicated; it first picks the nth page, in this case the first, before marking it complete. Yes, we could push that into one line but would forsake readability. If we run the wizard, we should be able to jump through the pages!

How Do I Love Thee? Let Me Count The Ways

True, there’s not much to the pages. Let’s change that by adding a spinner or two to the page. If you’re new to GTK+ design, everything is placed in a container widget, usually a horizontal or vertical ‘box’. This box acts like a stack in which objects are… well, stacked. Each slot can house only one object, though.

At present, each page is made up of a label. Let’s delete the second one to reveal an empty, grey area. After selecting the horizontal box from the palette, click in this grey area to place it. We’ll use the default 3 slots in the box, inside the first and last we can put a label. Change the text of these to something like:

Select from the spinners the first and last lines to display.
Click ‘Next’ to show the poem.

Perhaps a slight give-away as to what the final page will contain!

Boxes can be put inside boxes; by placing a vertical box in the second slot, we will house two spinners. You can play with their and any other object’s appearance with the ‘Packing’ tab in the properties. Lets set the upper and lower limits of the counter to zero and fourteen, as the lower and upper limit respectively.

The result is just around the corner (quite literally) as we create the lines of the poem to display. There are a number of ways to do this but we shall use a simple one: remove the label and replace it with another widget, a text view area. Set the text on this to the whole poem. Just copy the following in to the text view properties editor:

How do I love thee? Let me count the ways.
I love thee to the depth and breadth and height
My soul can reach, when feeling out of sight
For the ends of Being and ideal Grace.
I love thee to the level of everyday’s
Most quiet need, by sun and candle-light.
I love thee freely, as men strive for Right;
I love thee purely, as they turn from Praise.
I love thee with a passion put to use
In my old griefs, and with my childhood’s faith.
I love thee with a love I seemed to lose
With my lost saints, —- I love thee with the breath,
Smiles, tears, of all my life! —- and, if God choose,
I shall but love thee better after death.

Yes, that is Ms. Browning’s famous poem. This should size the application window to a suitably usable format now.

Our final addition to the program puts the spinners into action: add into the prepare_page method the following:

poem = @glade["textview1"].buffer.text.split("\n")

case @glade["assistant1"].current_page
    when 2
        poem_lines = " "
        for num in (@glade["spinbutton1"].value_as_int - 1)..(@glade["spinbutton2"].value_as_int - 1) do
            poem_lines << "\n" << poem[num]
        end
        @glade["textview1"].buffer.set_text(poem_lines)
 end

This includes a correction, value_as_int - 1, as arrays start from zero!

Conclusions

So, there we have it. You should be able to run through the program, pick which lines of a poem we should display and, well, display them. Not a lot in itself but all the set-up and groundwork is done for further development. We are picking up quite a lot along the way about the Ruby & GTK+ relationship:

Within a GTK+ project one can have many windows, each are independent. This allows for designers to concentrate on the GUI without interference from developers because designers need not code, nor developers need design.
All that one expects from an interface in programming terms is provided by the GTK+ objects and events, and Ruby interface. This gives flexibility and power to applications.
There exist short-cuts such as @glade[“”] for the @glade.get_widget method, that make for very easy reference of GUI objects, thus Ruby/GTK+ programming quite enjoyable.

In this post, we have added events to our basic GTK+ application by using Glade³ to assign a method to each, aligning them to Ruby code ‘callbacks’. In the process we have interacted with GUI objects and done some simple processing and activities based on the results. This is only the beginning, perhaps we could improve it by fixing the backward-button-clicking bug, adding other widgets and trying to manipulate them.

Ruby/GTK+ Development in SliTaz: Part I

Fri, 27 Aug 2010 13:40:00 +0100

One of my recent adventures has led to packaging the bindings I promised, so you can now develop GTK+ desktop applications that are powered by Ruby! The package I created is for SliTaz Linux, which I’ll be using in this series of posts to get you up and running. We shall walk through the process to get a basic application designed, built and running with the help of a couple of tools. I came up against a couple of problems just setting it up but this was new territory to me!

Prerequisites

This should work with any distribution provided you have the following software installed:

Ruby (to write your code)
- I’m using 1.8.7, but it may work with 1.9. It’s up to the bindings to represent the GTK+ stuff in Ruby correctly.
GTK+ (to show your UI)
- I’m using version 2.16, others may work. It’s up to GTK+ to have what the bindings expect, so you may be able to get away with earlier versions depending on what you want; likewise, newer versions should be backward-compatible.
Glade (to easily construct a UI)
- This should come with the GTK+ development tools but you may need to install it separately. You should also have LibGlade, which should be included.
the bindings betwixt them
- usually a ruby-gtk2 package but this will vary. Search your package management system for them; the ‘big ones’ should have them. You will need this on any system on which you intend to run your project.
Gladex (to write a kick-start file)
- This tiny utility creates a kick-start file in Python, Perl or Ruby. We’ll use this only once and it’s a tiny package. Unfortunately this is not yet available in the SliTaz repositories but the Debain package converts perfectly. Gladex replaces use of a tool in the Ruby/GTK2 package that requires modifications of the Ruby file.

If you’re using SliTaz, start a Terminal window and switch to root user (su), then run these commands to install them and their dependencies:

yes y | tazpkg get-install ruby-gtk2 ruby-dev glade3 gtk+-devtazpkg convert gladex-0.4.1-linux-2.6-intel.deb && \
tazpkg install gladex-0.4.1.tazpkg

Designing with Glade³

I found the easiest way of getting a small application together is to start with a basic UI design, without being fussy. By starting with the front-end, you can keep in your mind the function of the application and thus a roadmap with which to start. So, let’s start up the interface designer Glade³ for a blank canvas on which we can create our masterpiece!

You should immediately notice the main designing window but we need to set-up the project first. A Glade³ project is not just one window (though it can be) but houses all the graphical elements that one application needs. Sure, you can create one project for an application, another for its About window and so on, but I think it more commonsensical to keep them all in one project.

The abstracted nature of the Glade³ designer means that each project must have defined details of the environment in which you intend to run your project. The Ruby/GTK2 bindings need to use the LibGlade format, the predecessor to the newer GtkBuilder. To do this, ensure that LibGlade is set in the Project Preferences window:

Other options such as the version of GTK+ needed to run the project are also defined here. It will be fine to leave object names unique within the project and image resources within the project directory for portabilities sake. Click Close to set those options and get back to the designer window.

Here you face two options: design the windows yourself, or pick a template design from the Toplevels area of the toolbox. We shall do the latter, to save a little time. Moreover, experimenting with both will no doubt be fruitful in the future, to discover the GTK+ stacking of objects and such, but we are covering enough to understand the basics. Let’s create a basic wizard-style dialog, with the Assistant template.

The basic framework for a three-page assistant is created, just with one click. You can even jump through the pages with the buttons! You should see it in the Project Inspector to the top-right of the window. Each page is one widget, in the default case three text labels.

Before we do anything, let’s make this window active by turning on Visible in the Common tab of the assistant Properties. This will show the window when the application is started; without it, nothing would happen!

Now we can customise the front page with a welcome message, in just five steps:

Activate the widget by clicking in its centre. The Properties should now change to reflect its selection.
Modify its name to something a little more descriptive, lbl_Welcome. You should see this reflected in the Inspector.
Fill in a welcome message by changing the contents of the Label property under the Edit label appearence: sub-heading. LibGlade uses just plain text by default, so don’t worry about formatting it; if you want to include basic mark-up, select the Use markup: option below. You may find it easier to click the button to the right of the field, to open a new editor window.
If you enter a long line of text, the window will expand to fit it. Split up the text by turning on the Maximum Width in Characters property and setting it to 50. Word-wrap the text by turning on Word-wrap mode just beneath.
Lastly, switch to the Packing tab of the Properties and set a Page Title. You can add an icon as a header image too, if you like.

You’ve just made the first screen! Let’s run it… but how?!

Letting Your Glade³ Design Live

We set the project as a LibGlade file, which affects the layout of the Glade³ file. Somehow you need to get Ruby to talk with this Glade³ file. Thankfully, it’s pretty easy and you need to do this once. Use Point Gladex to the Glade³ UI design and an output directory, and choose the Ruby plug-in. This will create a kick-start file ready to run your application. If you open the first Ruby file with a text editor, you should see this line that allows for Ruby and GTK+ to talk:

require 'libglade2'

This makes available the Glade³ and GTK+ functions to Ruby. This includes starting the UI design. Notice that there is no conversion of the UI design, rather it is linked.

Now you should be able to start a Terminal window and run your application:

ruby [name].rb

It lives! Well, nearly; you may find it a bit lifeless at the moment: there is currently no Ruby code to make the UI do anything! Thankfully, as you started the application through a Terminal window, you can press Ctrl+C to force-close the UI.

In this post, we have created a basic GTK+ application with a point-and-click designer, hooked it up to a Ruby file in which we can write Ruby code to power it. Next we can give more life to the application, including when closing its window! Until then, have a look in the Ruby and Glade³ files to see their basic structure and references, and have a think about how we can power the application with Ruby code.

Ruby, Gtk+ & SliTaz... Nearly!

Sun, 22 Aug 2010 23:30:00 +0100

Just shy of two days was enough to let me make the lives of Ruby scribblers using the ever-awesome SliTaz Linux distribution that little bit sweeter. Sure, anyone with Ruby and its add-on system RubyGems can run web apps via Ruby on Rails in a snap, but what about desktop apps? Well…

Coming soon to a package manager near you:

Ruby/Gtk+ !

The package is oh-so-very-nearly ready to be uploaded to the servers - I’ll just tidy up the build instructions a little and await a little feedback from the adventurous, before-hand. The only thing you devs will need to do is install the ruby-gtk2 package alongside the Gtk+-dev stuff and include one of these lines at the start of your Ruby code, depending on which bit you want:

require 'gtk2'

require 'libglade'

As I’m using Glade to design my interface for me (stop laughing) I also used the slightly older LibGlade library, which is being replaced by GtkBuilder, with a tool to link them together. It still works very well though, as I shall demonstrate in a couple of future posts on the subject.

Stay tuned!

Who Are You? And What Do You Want (to do with my stuff)?

Mon, 10 May 2010 21:18:00 +0100

Facebook has been hitting the headlines recently as users are enraged, frustrated and disappointed by the ways in which the company handle the users’ data. Well — according to Facebook’s Terms of Use the data doesn’t really belong to the user but Facebook Inc., the company behind the site. Go and have a quick read through them if you don’t believe me; you may be surprised at what you find…

Let’s start with the basics. Facebook’s Terms Of Service state that not only do they own your data (section 2.1), but if you don’t keep it up to date and accurate (section 4.6), they can terminate your account (section 14). You could argue that the terms are just protecting Facebook’s interests, and are not in practice enforced, but in the context of their other activities, this defense is pretty weak. As you’ll see, there’s no reason to give them the benefit of the doubt. Essentially, they see their customers as unpaid employees for crowd-sourcing ad-targeting data.

(Source: Gizmodo.)

Since the privacy settings were overhauled last year (and the year before, probably the one before that too), the Terms of Use have been altered slightly, step-by-step, with more and more control over the user and their data. The virtual-social hub of millions of teenagers, twenty-somethings and even the Silver Surfer has arguably turned from networking-do-gooder to personal-all-knower.

Sound harsh? Step back a little: your data on Facebook is your name, photos, friends, what school/college/university you attended, when you were there, what you send via message or post to your friends, what the say back… everything on the publicly-available website. I think this video says it best.

I’ve started using a free alternative, Fossunet. It is much the same comparison to Facebook as identi.ca (Status.net) is to Twitter in the micro-blogging world; the former being an open platform providing the same, if not more, functionality. Okay, Fossunet isn’t as developed as Facebook quite yet, but it shows a lot of promise. One could argue that only third-party application developers is what is needed to make identi.ca really steal Twitter’s thunder.

The real difference between corporate entities and the more open alternative is this: your data should be your own. We have this freedom in the real world; for example, when we introduce ourselves to a fellow dog owner in the park, we don’t expect to take our photograph and staple it to every lamp-post in town. That may sound silly but if you put information on Facebook for your friends’ benefit, Facebook can do exactly that. Not very respectful unless you happen to be running in an election.

By using free and open alternatives to the ‘because everyone else is’ mainstream, we can each ensure we are treated on our own terms. If you don’t like how they treat you, fork off. (Forking is the term used when a free product is duplicated by another person/company and run alongside, on their own.) We are even free to license our own content and data as we see fit, with the option of reproducing the whole site for ourselves if they don’t agree. The two sites here are licensed under slightly differing Creative Commons licences, like many on-line projects and creative media. This entitles you to have a look at how they work, fiddle with them on your own should you so wish, and even launch one of your own. I implore everyone to take a critical look at the services they use and consider: is there a better alternative? (Maybe not just in ethics and freedom, but perhaps features or even price!)

On Your Docs, Set, Sprint!

Fri, 30 Apr 2010 21:09:00 +0100

Internet giants Google run a Summer of Code programme, in which ‘students’ each work on an open-source project, guided by mentors from their selected project. It’s a fantastic idea with good momentum and hype behind it; 3,400 students in nearly 100 countries have been accepted since its 2005 inception.

On the back of this, an article recently asked where the “Summer of Documentation” was; this gave me an idea. My favourite lightweight Linux distribution, SliTaz, is a young but already-excellent product, but is in need of a little TLC. @jpeg recently posted on the SliTaz Forum that the handbook and cookbook have not seen quite as much attention as they deserve, especially with the release of the latest SliTaz 3. I’ve advised developers in a previous post to shout out how great their efforts are with (better?) documentation; SliTaz deserves to show the world how simple and easy-to-use/fix it really is and I think this is a perfect way to do it.

So… :-)

I propose a period of time this summer, say a fortnight to a month, where a group of us focus on the SliTaz documentation to ensure it is of as high quality as SliTaz itself. This sprint can be carried out by the community, without needing developers to track issues and bug-fixes raised, as is the case in providing support on the forum.

I envisage the process involving a couple of wiki pages, in which we discuss which articles could be written - from FAQs to technical package details. (The mailing list could further discussions, but that’s not quite as open and used to be really productive; similarly the forum is a bit of a mish-mash of support and discussions.) Each could be taken up by one or more members of the community and the actual contents outlined. This will ensure that nothing is repeated (unnecessarily?) but, more importantly, everything is covered and we have substantial, useful documentation.

Do you have some spare time in which you could help with this impressive project? Could you give even an outsiders’ view to the docs sprint; is there anything we’re missing? Please don’t hesitate to give me or any of the contributors a message of advice!

I mirrored this post on the SliTaz Community, and have received some very useful and encouraging feedback from members. A big thank-you goes out to everyone involved in the project! I highly recommend you trying out this up-and-coming (if not already there!) project to see what all the fuss is about!

I'm Free!

Fri, 30 Apr 2010 21:06:00 +0100

As a software developer, the issue of copyright and distributivity can be a contentious issue. Some coders want to keep total control over their work, others want to share it among users.

Usually, software developed in a proprietary manner; I define this as many developers working for one company, which hides the wares from anyone else. This one company wants to capitalise on their idea — and efforts — only for commercial gain. Secrets agreements or clauses in contracts Often binds the development team and their efforts to that one company; many developers work for just one cause, their employer.

I understand perfectly that this enhances the reputation of the company and arguably their value to the field for producing their wares. But why constrict the clever chaps (and chap-esses) to just that one company? They can show off their skills and hours to the world, while remaining affiliated with that company. Surely this is even more effective marketing than just quoting the millions of pounds made from purchases?

I license my code as free and open-source software (FOSS) so anyone can use obtain and use my work, however they want to. I do this for a number of reasons:

It allows anyone to use my work, without being restricted to a certain number of copies or, say, running it on a certain number of computers;
Anyone can include my work in theirs (as long as they reciprocate and attribute me!);
I can use others’ work and include it in my own, to solve problems or to add extra stuff;
Whoever is involved in the projects can gain exposure by creating such useful software, especially with the people to which it matters most;
Other people can provide support in forums, mailing lists, social networks and other resources more easily, by reading the code and the problem hoping to find the missing link between them;
The combination of exposure and support means that feedback and improvements can be provided from all people involved in its use, from end-users to senior figures.
The free software licensing ensures it is kept free forever.

My passion for and admiration of free and open source software has led me to join the Free Software Foundation, more specifically their European branch. I’m a proud member of this group as feel it also shows a true commitment to free software and my principles. By providing a financial donation equivalent to two pints of beer a month, I help sustain the Foundation and the awareness, industry protection, campaigns and communities it creates. I carry around with me the smart-card; on it I store some geeky stuff for encryption, signing and logging in to my computer (yes, that geeky!). None of that was why I like this card, though; it signifies that I constantly and consistently abide by my free principles. Each time I open my wallet to show off a business card, borrow a library book or just buy a loaf of bread, the bold colours and title stands out for all to see.

All the code projects I am part of are free and open. Many cost no money to obtain. I am proud to be part of each and every one, in any case. I do so on the condition that the company also recognise my commitments, totally without force. Spreading the ideas and principles of free software and openness among the computing community — or any other — allows people like me to share my ideas and work more freely.

Don't Miss DEFT: v5X Review

Fri, 05 Feb 2010 13:00:00 +0000

Almost two months have passed since Stefano Fratepietro released the ‘100% Italian’ forensics distribution, DEFT Linux v5X. With support to its development given by the Italian Information System Forensics Association, this 660MB+ Xubuntu-based distribution is one not to miss.

Start-Up

Similar to the high number of *buntu derivatives, the boot screen presents the option to start DEFT, run the Memtest utility or boot the hard disk after asking for the preferred language. No time wasted on the booting screens; plain-text from the 2.6.31-14-generic kernel scrolls by to a text-based prompt. Nothing appears out of the ordinary here but those with sharp eyes may notice this message displayed:

ramzswap disk size set to 127340KB
Adding 127340k swap on /dev/ramzswap0

This is not activating swap partitions available on the hard disks but creating one from the leftover RAM at /dev/ramzswap0. Also among common forensic issues with Linux booting, the output is confirmation that ext3 partitions — such as the CD image and hard disks — are mounted as ext2, even at this stage. This ensures that journals are not used to modify the partitions, as noted in my earlier post on CAINE v1.5. Rest assured that the booting process does not compromise its forensic reliability, as the hashing of disk drives with ext3, FAT32 and NTFS partitions surrounding its boot-up and shut-down yield identical results.

When the system has booted — noticeably snappier than other Live distributions, might I add — DEFT automatically presents the user with the root command prompt, with advice to type startx to get graphical. While admins everywhere may find this harsh, a low-memory computer may benefit from such a preference and many applications can still be used. One would imagine that should this be necessary, the text prompt would be a fall-back option anyway. Unfortunately no more information was given about this prompt, so the X graphics system was started without hesitation.

Desktop & Applications

The LXDE/Xfce desktop is presented without complaint, after one startx command. Despite the overwhelming control provided by the root user, the privileges are given to ensure system-level applications work effectively. The desktop is well presented and comprises the file manager, terminal, MountManger, CD writer and an ‘Evidence’ folder. I am unsure of this tactic to give the user a default location for collected files in memory: first-time users may incorrectly assume that the folder is permanent storage but conversely could lead to the exact opposite when it is obvious the folder is not stored on the users’ usual media.

The Computer Forensics menu contained applications one would expect to find, such as Autopsy, Guymager, Ophcrack, WireShark and a hash calculator, GHash. There were also a few applications I had not found in other distributions, specialist or otherwise, such as:

Gigolo
easily connect to remote file-systems (SSH, FTP, WebDav etc.)
SciTE
text editor
ClamTK
virus scanner
XPlico
comprehensive network traffic capture with an Autopsy-like interface

The inclusion of these applications allow DEFT users to perform more on-site tasks than with other distributions, extending its usefulness beyond simple acquisition, verification and light analysis. Future issues can be identified such as viruses, the transmission of network data and data collection from remote sources.

A number of user-friendly additions appear in the Preferences menu, such as easy network file sharing — prompting installation of services if they are not present — and desktop configuration. (If these are recent additions in the (X)Ubuntu base I cannot say, as I’ve very little experience with it.) The toolkit is further improved with support for AFF and EWF files and their utilities, LVM utilities and desktop recording. These applications and libraries make DEFT a highly mature and worthy toolkit for forensics and incident response.

It was the third desktop icon drew my attention before any other: MountManager is a QT-based file-system mounting application. Unfortunately this is not so user-friendly; littered with per-file-system options to select, it lists the storage media available to the system in a check-box heaven. While the DEFT developers can do nothing about this, the interface is nothing short of horrendous on any resolution below 1200 pixels.On my first run, I conducted a test to simply mount ext3, FAT32 and NTFS partitions from MountManager, as if on a suspect system. Unfortunately to no avail, without indication of why the partitions simply did not respond. My mistake was perhaps using MountManager’s default /mnt mountpoint, as the MM worked using /media instead. When the PCMan File Manager was used, however, the partitions were mounted with default R/W options in the /media directory. Quite why neither application defaults to read-only mounting is an issue hopefully answered in future releases and made with great warning to current users.

A point worth discussing seems to appear in all manner of contemporary operating systems: memory usage. After my trail exploration of DEFT, the free -h command told of a 400-megabyte usage of the half-gigabyte allocated to the virtual machine. This was confirmed after a restart where the output was still 325 megabytes by just the desktop and terminal. Both these figures seem unnecessarily high considering any acquisition or similar task will consume more than its fair share of resources. This did not affect normal running of the desktop and applications, they were performing to expectations at these levels. However when the virtual machine’s RAM was decreased to a lowly 256 megabytes, 240 was used; similarly, only 2 megabytes remained free from 128. This led to a sluggish desktop and near-impossible application experience on low-end specifications, compounded with the noticeably-increasing starting and working times. This is arguably unanimous with CD-based live distributions but still worth considering in situations where memory is not up to contemporary standards.

Conclusion

One would imagine the wide-ranging collection of software given in DEFT are of particular importance to the parties using the distribution; one can assume they need more than the generic acquisition and analysis of systems while remaining in the scope of live response. This is extended with the inclusion of extra libraries and more fully-featured tools. Moreover, DEFT now raises the bar set by the competition. These show a generous amount of time and consideration have been put into the development of DEFT. One could argue performance will be hampered at lower-specification systems, especially if any intensive tasks are completed, but this seems a weakness on many similar distributions. Despite this, its relevance to IR is commendable and has become a mature and highly useful distribution.

Pitfalls of mounting file systems - Suhanov Maxim [PDF]

Sat, 23 Jan 2010 12:06:00 +0000

Pitfalls of mounting file systems - Suhanov Maxim [PDF]:

This short paper identifies requirements of forensic Linux distributions regarding boot-time file-system handling. Maxim identifies a couple of issues with boot-loaders & initialisation scripts and compares leading products for their vulnerability.

FreeBSD: 101... and a bit

Tue, 22 Dec 2009 20:30:00 +0000

In the last post in this three-parter, we installed FreeBSD on to either a physical or virtual computer. Now we need to apply some basic configuration to adjust permissions, choose a graphical log-in manager and start the desktop. Don’t worry, though: it’s very straightforward!

We have a basic FreeBSD installation, ready and waiting to be started. If we do so, we are presented with the FreeBSD bootloader — and a lovely one it is too. The options are far from an enigma; proceed with the first (default) option. Cast your eyes over the aesthetically repulsive babble that it will spew.

After a short time, we will arrive at the log-in prompt. I hope you remember the password for the ‘root’ account because the first few configuration steps are administrative and need the elevated priviledges.

It’s So Funny, It’s Been On xkcd.

To perform certain administrative commands, the root account must be used, so normal users cannot mess with the system. If these commands were mistyped, though, running them under the root account can be dangerous. To add a little protection, we shall allow the other user accounts to temporarily obtain administrative privileges through the sudo command. To do this, we must edit the /etc/sudoers file. As in Linux, we can do so by using the visudo command, a wrapper to the vi editor. The file is fairly straightforward; press the ‘Insert’ key to enable editing and duplicate the line that reads as follows, modifying it according to your other username:

root ALL=(ALL) ALL

Press Escape to exit editing mode, and type the following command to save the file:

:write

This will write the new sudoers file. Quit the visudo editor by typing:

:quit

If all is well, you will return to the prompt without any error messages. If errors are displayed, check the syntax of the line and that no others have been accidentally modified by running visudo again. We can now logout from root, back in as the normal user and safely perform administrative commands simply by prefixing your command with sudo. Each time sudo is used, it will ask for the normal account’s password: this ensures commands do not elevate themselves by using sudo without your knowledge. Let’s see sudo in action right-away to edit an configuration file.

Getting Graphical

By now, you may be a little tired of all this command-line business! While it’s extremely useful to know some shell-fu, there is a more intuative way around your system! The X Window System (a.k.a. X11, or simply X) is the tool for this job — but needs a helping hand from the hardware detection systems before it can be started. How else will it know about your graphics card, monitor or mouse? Use sudo and the familiar text editor vi to add the following lines to the start-up configuration at /etc/rc.conf :

hald_enable="YES" dbus_enable="YES"

With the two services now starting with the system, X should be able to run. Quit vi to return to the command line. If you don’t want to reboot again, we can start them manually with the following two commands:

sudo /usr/local/etc/rc.d/dbus start sudo /usr/local/etc/rc.d/hald start

If the two services have started, you should be able to start X when you boot up the system! Should you do this, though, you may not receive quite the interface you expect: your desktop environment, or DE, is the style of interface with which you use your computer. Many fall between two, GNOME & KDE, both of which are available for FreeBSD. So far, neither is preferred as the log-in is performed in a text-based console, after which a user must start X manually. If you wish to use the graphical system for everything, we can change this by adding a graphical login manager to the start-up sequence. This enables users to log-in with a pretty graphical system rather than a command-line and start X manually. To do this, edit the /etc/ttys file and find the line starting with “ttyv8”. If you have installed KDE, use KDM as in the following line; if you have GNOME, use GDM as in the 2nd line:

ttyv8 "/usr/local/kde4/bin/kdm -nodaemon" xterm on secure

ttyv8 "/usr/local/kde4/bin/gdm -nodaemon" xterm on secure

If GNOME is your thing, it might be a good idea to speed up GNOME’s loading mechanism by adding an extra line to /etc/rc.conf as before:

gnome_enable="YES"

That step isn’t necessary for KDE users; it just starts a couple more services behind the scenes to let GNOME start a little quicker.

The GNOME desktop environment is default with GDM but if you wish to just use the KDE desktop with either log-in manager, add KDE4 to .xinitrc in your personal (home) directory:

echo "exec /usr/local/kde4/bin/startkde" > ~/.xinitrc

That’s it! After you have rebooted your system, your system will have gotten graphical! Next up, we need to fit the desktop with all we need for a great forensic environment…

Happy Christmas and New Year to all my readers; I hope the winter break is as relaxing and happy as you hope to set you up for 2010!

CAINE 1.5 - "Shining" Example of Concious Development?

Fri, 27 Nov 2009 14:30:00 +0000

Four weeks ago I reviewed CAINE v1.0, the first full release of the Italian computer forensics LiveCD. It took the development team a mere six weeks to release version 1.5; how much an improvement is this edition? How far can one distribution go in such a short time to reach such an increment?

Well, should you look at the ChangeLog, the answer would be not very far. The changes listed are just additions of small software packages, without improvements to its application that one may expect from such a large version increase. A few improvements have been made, such as a ‘read-me’ in the ‘Bash Scripts Tools’ folder, a more complete copy of documentation from the website & a couple more icons are present on the Desktop — but aspects of CAINE’s application remain the same, such as its prized CAINE Interface. So too is the back-end of the system; I cannot notice much (if any) improvement on start-up or operational speed, system optimisations or further customisation of Ubuntu software.

While it is still a good product and contains many useful features, I fear that more time should have been taken to release this next iteration and a more accurate version number of v1.1 should be used due to this lack of major development.

Computer Forensic Analysis in a Virtual Environment -- Bem & Huebner, IJDE [PDF]

Mon, 23 Nov 2009 14:06:00 +0000

Computer Forensic Analysis in a Virtual Environment -- Bem & Huebner, IJDE [PDF]:

Discusses the use of virtualisation to facilitate analysis of imaged suspect storage. While the virtualised environment differs greatly from areal-world set-up, this method can prove powerful for live file analysis and/or re-creating the basics of the system.

One Step to Admiration and Appreciation

Sun, 22 Nov 2009 15:50:00 +0000

So often doing something is simply not enough. You need to show what has been achieved, talk about it, point out how other people can use it, build upon it. Most importantly of all, though: write about it.

Words on paper cement your actions and efforts for everyone to admire. This is more true in the digital world than ; describing your code is arguably more important than writing code in the first place. After all, if it weren’t useful, why should anyone use it? Why would anything be done if it didn’t have end results? It may seem drudgery to spend a while writing up what you have spent the past while looking at, but it really does take a very short time when compared to the length of the contribution. Quite how so many developers labour over their projects for days to years, with the only hint on how it works shrouded in the programming syntax of the source code. This is nothing short of perplexing! Even for those that do not fit the stereotypical quiet, modest worker, proverbially blowing one’s own trumpet should be done on every worthy occasion!

Found a niche or don’t like how other software solves a problem? Say so!

Come up with a neat trick in your code? Borrowed some methodology from elsewhere? Brag about it in a blog post! Scribble a note in the “how awesome is my software?” section of the documentation.

Wait, you haven’t written any documentation? Get to it! No-one will know how important your work is unless you tell someone something about it!

Let’s take a moment to think about the broader picture. When you contribute something to a project, big or small, who will be affected by it?

You.

This may be obvious but it goes deeper than one may expect: it doesn’t matter if you haven’t spent countless hours on a major project or merely minutes writing a small but useful patch (heck, they’re all useful if they’re good and even the bad ones are a start), you have spent time contributing to something. That’s time that could have been spent elsewhere. If it was coding, you will no doubt receive some kind of reciprocation for your efforts, be it a user thanking you or a member of the development team saying that it’s not how things are usually done and explaining how it should be. Your name will be noticed (believe me, it will) and it’s something you can use when trying to introduce yourself to other projects.

Developers / Artists / Musicians / Writers…

For other members of the project, it’s work they won’t have to do! Their project will be improved and it will be directly because of you. Whether it be code, a new icon, a sound-bite or additions/corrections to a website, the quality will be raised.

The World

What? I’m serious: your contribution may make your project that little bit better but that in turn raises the average. The little part of the world where your project sits has just become all the more special. The open-source movement has taken some flak for being weak in certain areas, a statement with which I agree, but conversely that means there are areas in which it succeeds. Its strengths would not be this great if people had not done something to make it so. Raise both the quality of your project and the bar for everyone else.

Users

This is both the biggest and smallest point. There will no doubt be a great percentage of users that will never be known. They stumble across the project, decide to try it out from its blurb or feature list. They may use it regularly or keep it somewhere in their system for that one ‘just the job’ moment. There are others however, that get in touch — notes are left on the project or developers blog, posts are sent to a mailing list, comments added to a bugtracker. But how did these users become so in the first place? I’ll gave you a hint: they found your project because they wanted it to do something, because it does something they want, because someone has said it does.

Gotcha!

Someone has said it does. Someone has written something, somewhere, to say what the project is and perhaps its features. Hopefully, there will be some kind of specification for its use too, its requirements or prerequisites. These are perhaps the second pieces of information that comes out of a project, as most places that house projects would require some.

For some though, this is how far the documentation goes. If this is the case, it’s not finished. In other areas of production, this level of information isn’t just a bad idea, it’s illegal. Buy a pizza from a supermarket and it may have a delicious name and describe what a ‘deluxe’ topping actually is but it is a legal requirement to give other information. Okay, in software there is no such definition to documentation but it is highly expected when delivering high-quality products.

Some potential users, though, only use software that is made in a certain way, uses certain other software or won’t do certain things. How can they decide and then become actual users?

I’m not just talking about the ‘stage one’ documentation, though. While step-by-step installations instructions and a good README are essential to get up and running, truly useful documentation takes much more. Great examples are real-world tutorials, answers to frequently asked questions and a plain English explanation of features, not a technical list of all the ins-and-outs of the interface (while that has it’s place). Expect your users to want to understand your project rather than just click, “Go.”

While the argument that a good user interface should be easy enough to understand in order to use the project, it just isn’t enough. Why does that button perform that event? Where does this list of data come from? How can we change it and why should we want to? While there is no common line to adjudicate for how detailed your docs should be, all of them should be practical, useful words on getting the best out of your project. It may take time, but so too does writing the code and everyone will appreciate it tenfold should there be notes explaining what it does and how to use it in the broader sense.

How much documentation do you write? Do you only use software or real-world artefacts that come with decent instructions, an FAQ and a help or support mechanism? Do you think that if someone can’t figure out how to use something, they shouldn’t? What are the best examples of documentation that you have seen? Let’s make everyone’s efforts shine all the more!

(Big thanks to Carla Schroder’s blog post on Linux Today for inspiring me to write this!)

CAINE v1.0 Released & Reviewed

Fri, 30 Oct 2009 18:30:00 +0000

It may be said that Linux distributions are like buses: we can wait at the roadside and see many interesting things go by, when we are waiting for our favourite to come around the corner it seems an age and we worry we’ve missed something, the old adage that we wait and - eventually - many turn up at once. No more true is that than now because Ubuntu has just had a new release, Fedora is currently in beta awaiting its finishing touches and, our feature presentation, CAINE has just turned the big one-point-zero.

Summary

Italian-born “Computer Aided Investigative Environment” is an Ubuntu-based Live CD for collection, analysis and reporting, featuring many common tools to form a decent forensic toolkit. Most notably a straightforward graphical front-end is included to guide users through the stages and - crucially, usefully - log the output of the called applications. Also noteworthy is the project’s Netbook edition, based on the USB edtion. At the time of writing however this was unavailable, so this review concentrates on the (perhaps greater used) Live CD version. I assume that my reader is familiar with the acquisition process and has previously used some of the applications included, such as dc3dd or AIR, as emphasis is not placed on them but the environment in which they are run.

Starting

The initial screen presented by CAINE is a customised GRUB boot-loader; it presents options to run the Live environment (in both full and safe graphics modes), install it using the standard Ubuntu installer, check the media for errors or continue to boot the computer as normal. I make a point of this because of one cosmetic feature: when an option has been selected, the menu appears to fade out to black. Tiny and insignificant, certainly, but its there in case you missed it. Other than this, the boot-loader… ummm… works. Good.

For some reason, the media check procedure is a hastily-added Crunchbang Linux feature, as the aesthetically-displeasing graphics show. I feel a simple text-based interface would have sufficed, as a defective disc is a defective disc; one could not do much to fix it.

Traditionally, Ubuntu(-based) distributions are hesitant to flood the user with messages as the system loads; they are often hidden behind a graphical splash containing a loading bar and a handful of text. CAINE refutes this, and adopts the more techie-friendly method of line-by-line output. Perhaps this is a good thing, but it may seem more polished if added.

In fact, it is a good thing for one simple reason: it allows a user to see what is going on and thus the boot process can be verified. To prove this point, one will notice a peculiar message is displayed, reading:

EXT2-fs warning (device: sda1) ext2-fill-super: mounting ext3 file-system as ext2

“Ahh! Mounting?! No!” may be your thoughts. Fear not; this simply means that the ext3 driver will be ignored when ext2 and ext3 partitions are mounted in the future and the ext2 driver used instead. This protects any ext3 partitions from a forensic point-of-view. Why? ext2 does not use journalling, so when an ext3 partition is mounted, there is no danger of modifying the meta-data when increasing the count inside said journal.

While we are on the subject of partitions: when they are mounted, the flags include ‘ro’ (read-only) and ‘loop’ (the loop-back device is used). The advantage of the ‘ro’ flag should be obvious but ‘loop’ perhaps not. By mounting an image file through the loop-back device, it can be treated as a physical disk. Another layer of read-only protection can be applied to the loop-back device. Furthermore, if a physical drive/partition is used, it should protect the meta-data. (I have not yet tested this, so if please provide me with a more correct/accurate explanation!) While this is effective for image files, take heed with physical partitions:

Warning: When using a loop-back device, occasionally the ext2 file-system seems to get corrupted for no apparent reason. If you start getting errors in the middle of make_root_fs from sys() complaining that a directory doesn’t exist, this may be what is happening. I don’t know why this happens and I can’t reproduce it consistently; however, unmounting, deleting and re-creating the file seems to get rid of the problem. Source (from 1998, I cannot be sure how correct this is!)

I have been informed that this v1.0 release is the first CAINE to be fully forensically sound. This is due to modifications applied to the start-up procedures (the init scripts) and patches applied to file-system drivers. I have not yet tested this distro to as fully a practical extent but hope to do so soon. I would imagine that the 1.0 milestone would b the perfect place to ensure such a claim is upheld!

Desktop

As in previous versions, the desktop interface is a customised GNOME, with a very Windows-like layout. The useful inclusion of the disk mounter applet, shown at the bottom-centre of the screen, is a list of partitions available on the system. Right-click on those icons and you can mount it read-only. One bug surfaces here: upon mounting, the icon is duplicated. I suspect this is because the device holds both a ‘sda1’-type node and a loop0-type node, because of the use of the loop-back system. This does not matter on a practical level however because the node at ‘sda1’ is still perfectly accessible and is still applied through the loop-back system. The menu is boosted by a ‘Forensic Tools’ folder containing links to the range of applications bundled (described later). Many standard desktop applications are provided, including word processor (AbiWord), spreadsheet (Gnumeric) and sound & movie players. A decent inclusion lies in gtkRecordMyDesktop, which creates a video of the on-screen activity - useful for documentation and evidential proof. On the technical front, a partition editor (GParted) is included along with a network manager (wicd) and… not much else.

Applications

The Forensic Tools folder is where the action is, quite literally. It is these applications that the distribution specialises in and the vast majority are controlled by a leading feature: the CAINE Interface. This GTK+ application guides the user through the stages a forensic investigation may take. A tabbed interface appears after we create the case and investigator name; each tab groups actions into image analysis, collection, system analysis and reporting. Each of these tabs simply launch various standard forensic applications and collects their output behind-the-scenes to present in a well-formed report. It may be a simple front-end that doesn’t do a lot of work but it doesn’t need to; it simply needs to collate the output of each program and attach some personal notes, which it does very well. In my quick testing of image acquisition and analysis, I could not find any problems with the end RTF-formatted report. In fact, if the application is kept open and parts of the process are completed after the report is generated, this is noted by adding revision information.

The applications launched combine to create a good range of features. Acquisition can be performed using either a GTK+ or Qt application, to local, network or removable media in a variety of formats; the analysis tools carry out interesting operations that may yield effective results. They provide a decent level of logging - some even the command executed - to show activity, a standard practice if not requirement.

There are a few problems with each application but these are arguably outside the concern of CAINE’s developers. For example, AIR displays only IDE disks in the menu but both IDE and SATA in the toolbar (which rests at the bottom of the window, for some UX-related reason?) when they should reflect each other. Guymager automatically appends an extension to the image file, compromising some flexibility. Documentation could be improved - arguably across the board - too.

One area of concern lies in the folder named, “Bash Scripts” which contains a handful of scripts to run inside a terminal window. At a quick glance, I cannot work out what they are for and some of them do not work (the ddrescue and dc3dd contained in this folder are not even scripts but links, and return the error, “Too many levels of symbolic links,” on both display and execution). These are linked to both on the desktop and in the launcher menu, so one can assume they are of some usefulness or importance. One can run the commands, referring to those stored in /usr/bin etc. but then I cannot see why these links exist!

As with many Live CD environments, memory usage is an issue. By running the free command inside a terminal window as soon as the desktop has loaded, we can see that nearly five hundred megabytes of RAM is used. Of course, swap is not activated either, so this cannot compensate. On my test system, this left behind six for my applications! One would imagine that this adversely affects the performance of particularly the acquisition and hashing operations but I have not yet performed such tests. This will however decrease should the product be installed, an action I did not take during my testing.

Conclusion

The first full release of CAINE fulfils nearly every requirement for an acquisition, analysis and reporting platform. It is apparently the first release to be fully forensically sound, thanks to the modifications applied to the start-up procedures. One may forgive the perhaps heavy desktop as it is a familiar one, but could argue that alternatives could be used satisfactorily. The applications included are all stable, useful and included for a purpose. One would hope that optimisation is on the developers minds for future releases.

Post Updates
1st CAINE release to be fully forensically sound (31 Oct ‘09)
init scripts & fs driver modifications (31 Oct ‘09)
disambiguation (31 Oct ‘09)

FreeBSD: 101

Thu, 22 Oct 2009 10:40:00 +0100

The majority of content here focusses on the GNU/Linux operating system because it is freely available to obtain, install and use. An alternative lies in its perhaps lesser-used cousins, the BSD family. While slightly more technical, they are an equally ultra-reliable bunch. If Linux isn’t quite your thing, you may wish to use FreeBSD instead. It’s good idea to at least acknowledge these alternatives exist so I’ve put together a quick How-To guide to get you started using FreeBSD. I do so using virtualisation in Sun’s VirtualBox software, but it makes no difference in terms of the end result.

I aim to provide a guide to get FreeBSD up and running more quickly than following the FreeBSD Handbook, an excellent resource should you get stuck. These posts are for those coming from using Linux, so will assume a decent knowledge of Linux and that you’re not afraid to learn! This post runs through the installation routine; later posts applies some important post-installation configuration to give a solid base system. (We do not, however, discuss any troubleshooting.) An outline of the OS will follow but there’s only one real way to find out, right?

Grab FreeBSD (the easy bit)

FreeBSD is available from the website; the only necessary CD-ROM image is #1. Either burn it to a physical CD-ROM or — if you’re using virtualisation — create a virtual machine with it as the CD-ROM. I estimated that a clean 10GB hard disk will be plenty to house an installation of version 7.2. With your environment prepared, the next step is to install FreeBSD by booting the CD-ROM.

The Installation (a.k.a, “I wish I had a mouse”)

The installation is fairly painless, though it may appear a world away from the point-and-click systems that many current operating systems feature. To navigate, use the arrow keys and the space-bar to select. This is the standard method to use an ‘ncurses’ or keyboard-based program.

After selecting your language settings, chose the ‘Custom’ installation method because it is a balance between having few and too many options. This presents a menu structure which you should take one step at a time, down the list. he Options section can be skipped as the only settings inside worth changing can also be set later.

Partitioning your drive can be as simple or as complicated as you wish; simply tell the installer to use the entire disk as a ‘slice’ (press ‘A’), make it bootable (scroll down to it and press ‘S’) before finishing the first partitioning stage (with ‘Q’). Since your drive only houses FreeBSD, you can leave out the bootloader by selecting ‘Standard’ from the next prompt.

Inside the ‘slice’ partition, you need to create the secondary partitioning structure that defines whaere parts of the file-system are stored. The default is acceptable, but I simply create a root (‘/’) partition filling the whole disk, minus a 512MB swap at the end. To do this, hit ‘C’ and specify the size of your first partition (in my case, 9727M), its mountpoint (/) and allowing SoftUpdates upon it (’S’). I repeated this to create the swap in the remaining space. By finishing this, the disk can be prepared for software.

Distributions are essentially bundles of software for types of users. This quickens the installation process substantially. I selected ‘X-User’ with the Ports collection by choosing option 9 and ‘Yes’ at the following screen. The Ports collection is by far the easiest method of software installation; it is the equivalent of the ‘apt’ or ‘yum’ package management systems available on Linux. If you do not install this, you will have a lot of compiling to do when it comes to installing software later!

The installation will commence after the media type is selected, in this case CD-ROM, and your choices confirmed. The disk drive will be prepared, base files extracted and your chosen software installed upon it.

The Installation… The Other Half

Now that the software has been installed, you will be asked if you wish to perform some additional configuration. Proceed to do this because it contains some important steps.

A very useful package lies in the ‘linux’ group: Linux binary compatibility allows… well, compatibility with Linux binaries. That is to say they will run under FreeBSD.

Another utility that Linux users will want is sudo, found under ‘security’. This allows regular users to run administrative commands and will need to be configured later to do so. While there is no necessity to use sudo, it is a good habit to do so for security reasons.

Installing the two packages inside the ‘ports’ group will assist software installation. This will also be discussed later. For now, select both packages there and stop making such a fuss.

For a graphical interface, we’ll need to install some additional packages. While the Distribution contains packages for graphics, we don’t yet have a desktop environment. By selecting the ‘kde’ group inside the package list, we are presented with each component of KDE v4.2. While you could choose another desktop environment from ‘x11-wm’, I like to use a more feature-filled option, such as KDE or GNOME. KDE is available by the ‘kde4-4.2.2’ item, GNOME from ‘gnome2-2.26.0’ — these are meta-packages, bundles of software which will group the necessary components together in one selection. Notice that other packages are automatically marked for installation as dependencies, with a ‘D’.

With all the additional software packages selected, continue to install them by pressing ‘Tab’ to select ‘Install’. Depending on the desktop slected, this stage will often seem as long as the initial set-up…

Post-Installation Settings

Now that the package selection is complete (at last) the last few settings can be carried out. We wil continue to take the next menu step-by-step:

Arguably the most important setting is the Root Password: the administrative account, named ‘root’, is the only one from which system-level operations can be successfully carried out. Choose a strong password here so the computer will not be easily compromised. Hopefully you won’t need to use it that often, so pick something you’ll remember!

The next two option, ‘Fdisk’ & ‘Label’ we have already completed in the earlier partitioning, so adding a normal user is the next step. Fill in the login ID (user name) field, a password and enter your full name. The other fields can be skipped.

The only remaining options to set are ‘Time Zone’, ‘Mouse’ & ‘Networking’ — self-explanatory and, because they are computer-dependant, outside scope. Use your knowledge and intuition. After those are set, we can finish the installation by selecting ‘Exit’, the uppermost option on the ‘Configuration Screen’ list and the ‘Exit Install’ button. The installation system will close and your prepared computer reboot.

Conclusion

So far, the basic FreeBSD software have been installed to the hard disk. Those few additional packages expand the selection to give a solid base from which we will continue to use it. Lastly, a little administration was applied to personalise the system. In the next post, we shall boot our installation and perform a little user-level configuration to get things running even more smoothly.

Harden a Linux Kernel

Thu, 24 Sep 2009 14:33:00 +0100

At the core of any operating system is its kernel, the basic software code that manages system resources and where all code between application and computer passes through. One can imagine that as this software is at the most basic level, it is a prime target for exploitation.

This disadvantage can be turned around, however; by introducing or imporving security measures at this level means that it can be an effctive barrier on all later levels. Many projects exist to do so:

Security-Enhanced Linux from the NSA is built into common desktop Linux systems. Gaining popularity through the Fedora Project, SELinux is available for Linux, FreeBSD, OpenSolaris and Darwin (Mac OS).
The Linux Intrusion Detection System (LIDS) is a patch applied on top of the Linux kernel and provides security through rule-based access control. It suppresses the all-access power of the super-user (root) while so limited damage can be done to the system. It also protects itself through a strong password authentication mechanism.
RSBAC has been implemented in to the Linux kernel since 2000 and provides access control, similar to LIDS, along with other goodies.
Finally (though I suspect many more exist) grsecurity is aimed at web servers or those that accept remote connections. Emphasis is placed on buffer overflows and other more low-level vulnerabilities.

Excellent Trick on JPG Images. An Image That Contains Images | Jeez Tech

Wed, 23 Sep 2009 02:49:00 +0100

Excellent Trick on JPG Images. An Image That Contains Images | Jeez Tech:

Interesting tutorial to create one file that is both a JPEG and a ZIP.