Using H2 In-Memory to test your DAL

How should we test the Data Access Layer code?

Many developers ask that question. Similar to the other layers of your system, it should be fully tested to prevent unexpected and random behavior in production. There are many ways to achieve that, among of which are mocks or in-memory database.

The problem with mocks is that instead of testing the validity of your SQL queries (syntax and execution), you will only be testing the validity of the system’s flow. On the other hand, using the in-memory database will validate both! That is why I prefer it over mocking. But, you should keep in mind that the SQL syntax might differ from one database engine to another.

In this blog, I will be giving an example of using “H2 In-Memory” in unit tests. The code of this example is available on my GitHub account.

H2 In-Memory in Action

So, let us see H2 in action 🙂

For the sake of this blog, I will assume we want to test three SQL operations on the table “Members” having the following model:

MEMBERS
ID      | NAME        |
INTEGER | VARCHAR(64) |

The SQL operations to be covered in this blog are:

  1. Create Table
  2. Insert (batch of prepared statements)
  3. Select *

Code Explanation

The example will be based on five files (pom.xml, Member.java, SqlRepository.java, H2Repository.java and H2MembersRepositoryTest). In this section, I will give a brief explanation of each file.

Pom.xml (Maven Dependencies):

First, let us modify our pom file (as shown below) to make our project depend on two projects:

  1. com.h2database: Using this dependency, Maven will take care of downloading the h2 jar file we will be referencing in our tests.
  2. Junit: a unit testing framework for Java
<dependencies>
    <dependency>
        <groupId>com.h2database</groupId>
        <artifactId>h2</artifactId>
        <version>1.4.191</version>
    </dependency>
    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>4.11</version>
    </dependency>
</dependencies>

Member.java:

This class represents a Member record. It consists of a factory method that returns an instance of the Member class and two getter methods to return the values of the Id and Name fields.

package dal;

import java.util.Objects;

public final class Member {
    private final int id;
    private final String name;

    private Member(int id, String name) {
        this.id = id;
        this.name = name;
    }

    public static Member aMember(int id, String name) {
        return new Member(id, name);
    }

    public int id() {
        return id;
    }

    public String name() {
        return name;
    }

    @Override
    public boolean equals(Object o) {
        if (this == o) return true;
        if (o == null || getClass() != o.getClass()) return false;
        Member member = (Member) o;
        return id == member.id &&
                Objects.equals(name, member.name);
    }

    @Override
    public int hashCode() {
        return Objects.hash(id, name);
    }
}

SqlRepository.java

In this class, we establish a connection to our database. What is important in this class is that we pass the connection string as a parameter to the constructor thus making our code unbounded to any specific database engine (MySql, H2, Sybase, Oracle, etc.). This will make writing our tests much easier!

The class consists of:

  1. Constructor: initializes an instance of SQL Connection using the connection string passed as parameter
  2. Three public methods:
    1. createTable: executes an update query to create the”MEMBERS” table.
    2. allMemebers: executes a select query and returns the found records in a list of Members.
    3. insertMembers: takes a list of “Member” as a parameter and inserts the values into the “MEMBERS” table.
package dal;

import dal.Member;

import java.sql.*;
import java.util.ArrayList;
import java.util.List;

public class SqlRepository {

    protected final Connection connection;
    private static final String CREATE_MEMBERS = "CREATE TABLE MEMBERS(ID INTEGER, NAME VARCHAR(64))";
    private static final String SELECT_MEMBERS = "SELECT * FROM MEMBERS";
    private static final String INSERT_MEMBERS = "INSERT INTO MEMBERS(ID, NAME) VALUES(?, ?)";

    public SqlRepository(String connectionString) throws SQLException {
        connection = DriverManager.getConnection(connectionString);
    }

    public boolean createTable() throws SQLException {
        Statement createStatement = connection.createStatement();
        return createStatement.execute(CREATE_MEMBERS);
    }

    public List<Member> allMembers() throws SQLException {
        List<Member> allMembers = new ArrayList<>();
        Statement selectStatement = connection.createStatement();
        ResultSet membersResultSet = selectStatement.executeQuery(SELECT_MEMBERS);
        while (membersResultSet.next()) {
            allMembers.add(Member.aMember(membersResultSet.getInt(1), membersResultSet.getString(2)));
        }
        return allMembers;
    }

    public void insertMembers(List<Member> members) throws SQLException {
        final PreparedStatement insertMembers = connection.prepareStatement(INSERT_MEMBERS);
        members.forEach(member -> insertMember(member, insertMembers));
        insertMembers.executeBatch();
    }

    private void insertMember(Member member, PreparedStatement insertMembers) {
        try {
            insertMembers.setInt(1, member.id());
            insertMembers.setString(2, member.name());
            insertMembers.addBatch();
        } catch (SQLException e) {
            throw new UnsupportedOperationException(e.getMessage());
        }
    }
}

H2Repository.java

I added this class under “Test Sources Root” because it is only used by the tests.

As you notice, it extends the SqlRepository class implemented previously. Thus, we don’t have a lot to implement here. The only method added is a new method “closeConnection” that drops all the existing tables from the database.

You might wonder why would we need that since we are using an In-Memory database. That might be true for this simple example, but it will be a necessity when running multiple tests classes. That is because, in Java, all the tests are run in the same JVM which means that the H2 instance initialized in the first test will be shared with the next test classes. This approach might lead to an unexpected behavior when using the same tables in the different test classes.

package dal;

import java.sql.SQLException;

public final class H2Repository extends SqlRepository {
    public H2Repository(String connectionString) throws SQLException {
        super(connectionString);
    }

    public void closeConnection() throws SQLException {
        connection.createStatement().execute("DROP ALL OBJECTS");
    }
}

H2MembersRepositoryTest.java

This is the simple test class! Our single test (it_correctly_inserts_members_to_a_database) is invoking the three methods we implemented before (createTable, insertMember and allMembers). If any of those methods is badly written our test would fail.

package dal;

import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Test;

import java.sql.SQLException;
import java.util.ArrayList;
import java.util.List;

import static dal.Member.aMember;
import static org.fest.assertions.Assertions.assertThat;

public class H2MembersRepositoryTest {

    private static final String H2_CONNECTION_STRING = "jdbc:h2:mem:test";
    private static final List<Member> MEMBERS = new ArrayList<>(10);
    private static H2Repository h2Repository;

    @BeforeClass
    public static void
    setup_database() throws SQLException {
        h2Repository = new H2Repository(H2_CONNECTION_STRING);
        initializeMembers();
    }

    @Test
    public void
    it_correctly_inserts_members_to_a_database() throws SQLException {
        h2Repository.createTable();
        h2Repository.insertMembers(MEMBERS);

        assertThat(h2Repository.allMembers()).isEqualTo(MEMBERS);
    }

    private static void initializeMembers() {
        for (int index = 0; index < 10; index++) {
            MEMBERS.add(aMember(index, "Name_" + index));
        }
    }

    @AfterClass
    public static void
    tear_down_database() throws SQLException {
        h2Repository.closeConnection();
    }
}

TearDown

If you can test your DAL, you can test anything! (I just came up with this ;))

Keep the tests going!

References:

  1. H2-Database Engine
  2. H2-Maven
  3. JUnit Maven
  4. GitHub

How to reference Ruby-JMeter from Java project?

INTRODUCTION

Apache JMeter is an open-source Java application that allows you to design, build and run performance tests. It is an excellent tool and currently we are using it to analyze our software’s performance. But being a graphical tool, after building a couple of tests you will notice that most of the test steps were repeated each time. It becomes frustrating and a waste of time and here is when Ruby-JMeter comes to the rescue!!! Ruby-JMeter is a tool that allows you to write your JMeter test plan in Ruby and then convert it to a JMX file. Since it is an open-source project, you have the flexibility of extending it and customizing it to fit your needs.

How can this tutorial benefit you?l

The purpose of this tutorial is to show how to integrate ruby-jmeter within a Java project. We are using this integration in an in-house Java based benchmark framework that dynamically reads and executes JMX files. Another way to benefit from this tutorial is if you want to reference any of the RubyGems from your application. Ruby-jmeter is one of the gems hosted on RubyGems, thus, you can follow the steps defined in the 2nd part of the tutorial to reference other gems.

Tutorial Structure

I have divided this tutorial into three sections. The first one shows how to execute ruby scripts directly from Java; the second section shows how to ruby ruby-jmeter scripts from Java. Whereas the last section is dedicated from some refactoring of our test class.

Executing Ruby Scripts

1. Maven Dependency

One of the possible options to run Ruby from Java is by using JRuby.
Since we are writing a maven project, then we need to add JRuby dependency in our pom. To do that we just add the following dependency to the dependencies section of our pom.Xml file.

<dependency>
    <groupId>org.jruby</groupId>
    <artifactId>jruby-complete</artifactId>
    <version>9.0.3.0</version>
</dependency>

2. RubyExecutor

Now we can reference JRuby in our code, so let us start by adding a new java file “RubyExecutor” to our project. In this class, we will add a static method that takes the full path of the ruby script as a parameter.

Below is the code for the method:

public class RubyExecutor {
    public static void run(String rubyFile) throws FileNotFoundException, ScriptException {
         final ScriptEngine scriptEngine = new JRubyEngineFactory().getScriptEngine();
         final FileReader rubyFileReader = new FileReader(rubyFilePath);
         scriptEngine.eval(rubyFileReader);
         rubyFileReader.close();
    }
}

The first line initializes an instance of JRubyEngine.
The third line just calls the eval method that is responsible for executing the ruby code in the JRubyEngine.

3. Testing our code

The best way to assert our code is working as we want, is by writing JUnit tests. In this case, we can add a new test class RubyExecutorTest.java under “src/test/java”.
For simplicity, we will make our ruby code generate a new file at the end of its execution. This generated file has the same filename but with “jmx” as an extension.
Based on that we should make our test call the method RubyExecutor.run() and assert that a new file has been added.
First we should start by writing a simple ruby script in the test-resources folder.

Steps:

  1. Add a new folder “src/test/resources/RubyExecutor”
  2. Add a new ruby file “simpleCode.rb” under that folder
  3. Add the following code to the Ruby script. This code creates a new empty file with the name “simpleCode.jmx”
`File.new('target/test-classes/RubyExecutor/simpleCode.jmx', "w+")`

Now we can write our test class that should look like this:

public class RubyExecutorTest {
    @Test
    public void
    it_should_generate_a_file_from_ruby_code() throws IOException, ScriptException {
       final String rubyFile = this.getClass().getResource("/RubyExecutor/simpleCode.rb").getFile();
       RubyExecutor.run(rubyFile);
       final File expectedFile = new File(rubyFile.replace("rb", "jmx"));
       Assertions.assertThat(expectedFile).exists();
    }
}

The test asks the RubyExecutor to execute the ruby script named “simpleCode.rb” (the one we created in previous steps) and asserts that the file “simpleCode.jmx” has been created.
Now let us try to run the test using maven command. To do so run the below command from the directory “jruby-jmeter-integration”

mvn clean install

The tests should be green, and we should get the following success message:

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:17 min

We should also notice that the file “simpleCode.jmx” has been created under “target/test-classes/RubyExecutor/”
Note: I use the library org.fest.assertions.Assertions, thus make sure you have added it to your dependencies in maven.

Executing Ruby-JMeter Scripts

Now that we are capable of executing Ruby scripts from our project, we will work on running ruby-jmeter scripts.

1. Ruby-JMeter

As stated earlier, ruby-jmeter is a tool that allows you to define your jmeter test-plan in Ruby code. The tool provides methods for most of the JMeter controllers, thus, it is a matter of calling the right method with the right parameters. Upon execution, the tool will generate a JMX file from the Ruby code.
I’m not going to dig into details on how it works, for more details you can have a look at some examples available on project’s workspace on GitHub

Now it is time to write a simple test plan with ruby-jmeter.

Steps:

  1. Add a new ruby script “src/test/resources/RubyExecutor/simpleJmx.rb”
  2. Add the following code to the Ruby script:
require 'rubygems'
require 'ruby-jmeter'   
test do
  threads count: 1, rampup: 0, loops: 1, scheduler: false do
    beanshell_sampler query:'log.info("********Hello World********");'
  end
end.jmx(file: 'target/test-classes/RubyExecutor/simpleJmx.jmx')

This ruby code will do the following:

  1. Create a new test plan
  2. Add a new “Thread Group” with a single thread and single loop count
  3. Adds a BeanShellSampler as a child of the thread group to print “********Hello World********”
  4. Saves the test-plan in the file simpleJmx.jmx

The generated JMX file should look like this:

SimpleJmx

2. Ruby-JMeter within our maven project

Now we know exactly what our code should do; what are its input and output. So, it would be great if we write a JUnit test to check if our code has indeed generated a JMX file from the Ruby script we wrote in the previous section.
So our test should execute the ruby script and is expected to generate the respective JMX file (“target/test-classes/RubyExecutor/simpleJmx.jmx”) at the end of the execution.
Based on this, we can add a new test to RubyExecutorTest.java

@Test
public void
it_should_generate_a_jmx_file_from_ruby_code() throws IOException, ScriptException {
 final String rubyFile = this.getClass().getResource("/RubyExecutor/simpleJmx.rb").getFile();
 RubyExecutor.run(rubyFile);
 final File expectedFile = new File(rubyFile.replace("rb", "jmx"));
 Assertions.assertThat(expectedFile).exists();
}

Let us give it a try:

mvn clean install

Unfortunately if fails:

LoadError: no such file to load -- ruby-jmeter
    require at org/jruby/RubyKernel.java:939
    require at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:54
        <top> at <script>:2
    Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 11.657 sec <<< FAILURE!
    it_should_generate_a_jmx_file_from_ruby_code(RubyExecutorTest)  Time elapsed: 11.446 sec  <<< ERROR!
    javax.script.ScriptException: org.jruby.embed.EvalFailedException: (LoadError) no such file to load -- ruby-jmeter
        at org.jruby.embed.jsr223.JRubyEngine.wrapException(JRubyEngine.java:104)
        at org.jruby.embed.jsr223.JRubyEngine.eval(JRubyEngine.java:121)
        at org.jruby.embed.jsr223.JRubyEngine.eval(JRubyEngine.java:146)

The exception is telling us that it couldn’t execute ruby-jmeter. Maven wasn’t able to locate the ruby-jmeter libraries as we haven’t referenced it yet in any of our code.
To fix this problem, we will have to add more dependencies to our pom file!

3. Ruby-Gems Dependency

RubyGems is a repository hosting Ruby community’s gems. Once referenced, Maven will access this repository to sync the required ruby libraries locally to your project.
In our case we are using the ruby-jmeter gems to add dependency on that we should modify our pom file as follows:

1.  Add repository: This will be used by maven to download the gems from the specified URL

     <repositories>
        <repository>
            <id>rubygems-releases</id>
            <url>http://rubygems-proxy.torquebox.org/releases</url>
        </repository>
    </repositories>

2.  Add dependency on ruby-jmeter

<dependency>
    <groupId>rubygems</groupId>
    <artifactId>ruby-jmeter</artifactId>
    <version>2.13.8</version>
    <type>gem</type>
</dependency>

3.  Add build step

<build>
    <plugins>
        <plugin>
            <groupId>de.saumya.mojo</groupId>
            <artifactId>gem-maven-plugin</artifactId>
            <version>1.1.3</version>
            <extensions>true</extensions>
            <executions>
                <execution>
                    <goals>
                        <goal>initialize</goal>
                    </goals>
                </execution>
            </executions>
            <configuration>
                <includeRubygemsInResources>true</includeRubygemsInResources>
            </configuration>
        </plugin>
    </plugins>
</build>

4.  Rerun maven:

mvn clean install

Now we should get a “Build Success.”

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running RubyExecutorTest
I, [2015-12-23T17:27:19.643000 #7564]  INFO -- : Test plan saved to: target/test-classes/RubyExecutor/simpleJmx.jmx
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.278 sec
Results :
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ java-ruby-jmeter-integration ---
[INFO] Building jar: D:\Dev\java\workspace\GitHub\sample-projects\jruby-jmeter-integration\target\java-ruby-jmeter-integration-1.0-SNAPSHOT.jar
[INFO]
[INFO] --- maven-install-plugin:2.4:install (default-install) @ java-ruby-jmeter-integration ---
[INFO] Installing D:\Dev\java\workspace\GitHub\sample-projects\jruby-jmeter-integration\target\java-ruby-jmeter-integration-1.0-SNAPSHOT.jar to D:\Dev\maven_repo\aatwi\github\java-ruby-jmeter-integration\1.0-SNAPSHOT\java-ruby-jmeter-integration-1.0-SNAPSHOT.jar
[INFO] Installing D:\Dev\java\workspace\GitHub\sample-projects\jruby-jmeter-integration\pom.xml to D:\Dev\maven_repo\aatwi\github\java-ruby-jmeter-integration\1.0-SNAPSHOT\java-ruby-jmeter-integration-1.0-SNAPSHOT.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------

Code Refactoring

The last part of this tutorial is to do some code refactoring to our test class. Although it isn’t essential here, it became a habit for me!!
If we look at our current code, we can notice that there is some code duplication in the two tests.
One thing we can do here is extracting the method “assertJmxFileExists” to remove this duplication.
Our new method will take the responsibility of calling RubyExecutor.run and asserting the generation of the JMX file.
The code below shows the difference before and after refactoring.
Before Refactoring:

@Test
public void
it_should_generate_a_file_from_ruby_code() throws IOException, ScriptException {
    final String rubyFile = this.getClass().getResource("/RubyExecutor/simpleCode.rb").getFile();
    RubyExecutor.run(rubyFile);
    final File expectedFile = new File(rubyFile.replace("rb", "jmx"));
    Assertions.assertThat(expectedFile).exists();
}  

@Test
public void
it_should_generate_a_jmx_file_from_ruby_code() throws IOException, ScriptException {
    final String rubyFile = this.getClass().getResource("/RubyExecutor/simpleJmx.rb").getFile();
    RubyExecutor.run(rubyFile);
    final File expectedFile = new File(rubyFile.replace("rb", "jmx"));
    Assertions.assertThat(expectedFile).exists();
}

After Refactoring:

@Test
public void
it_should_generate_a_file_from_ruby_code() throws IOException, ScriptException {
    assertJmxFileExists("simpleCode.rb");
}

@Test
public void
it_should_generate_a_jmx_file_from_ruby_code() throws IOException, ScriptException {
    assertJmxFileExists("simpleJmx.rb");
}

private void assertJmxFileExists(final String rubyScript) throws IOException, ScriptException {
    final String rubyFile = this.getClass().getResource("/RubyExecutor/" + rubyScript).getFile();
    RubyExecutor.run(rubyFile);
    final File expectedJmxFile = new File(rubyFile.replace("rb", "jmx"));
    Assertions.assertThat(expectedJmxFile).exists();
}

We can do more refactoring, but I think it is not worth at this point. The code is simple and can be easily read and understood.
Finally, rerun maven to make sure that nothing was broken with this refactoring.

References

Let’s make the tests green

Two weeks ago we started running coding dojo sessions at our offices in Beirut, the first sessions we had so far were much better than what we expected. In this article, I will be sharing my experience (as the one running those sessions) starting from the preparation and ending by the feedback on those sessions.

 

What is CodingDojo?

 
CodingDojo is a meeting where a group of people gets together to solve a programming challenge following TDD. We select those challenges from several online sites like Google Code Jam and Cyber Dojo.
Each dojo session is planned to take 2 hours including a 15-minute retrospective at the end and is scheduled once every week.  
All the participants are contributors, as we rotate on the driver (one who is coding) every 5 minutes. Thus, it is important to limit the number of participants per session to a maximum of 8
We carefully chose the session timings to be during lunchtime from 12:00 to 14:00 in order not to interrupt participants’ usual tasks. 
Finally, it is important to note that the participation to those sessions is not restricted to specific teams, rather anyone regardless of their programming expertise is welcome to join. After all, codingdojo is a place where we take knowledge sharing to the extreme. 

 

From Paris to Beirut 

 

It started almost 2 years ago as an activity within our team members only, but 8 months ago it was changed to involve interested people from other teams (in Paris only) as well. By the end of each session a 15-minute retrospective was held, during which we have always received positive feedback from the participants. For them, it was a chance to learn how to write code following TDD, learn new programming languages and techniques and enjoy a teamwork spirit to solve a programming challenge.  
We decided to leverage on the experience we gained in Paris and start similar sessions in Beirut. At that point, we had no clue on the participation level or its success rate but it was a risk we were willing to take. 

                     

 

Preparation 

 

The first step of the preparation was to contact the HR team asking them for sponsorship as such an activity involves communication and involvement of many teams. We had a meeting with them, explained what codingdojo is and what is expected from those sessions; as expected, they were very supportive to such an initiative. They granted us their sponsorship and offered their help in global communication, coordination with the administration to provide lunch for participants and whenever needed later. 

Second came the technical preparation for better running and managing such an activity. Writing code following TDD became a habit as I have been doing that for more than a year and a half (since I joined my new team). But still I decided to practice more by solving some challenges outside working hours.
For me managing such sessions was the main challenge and a new experience. For that, I started attending (via Visio conference) the sessions managed by Philippe (my teammate in Paris). During which, I tried to benefit from Philippe’s experience and save tips and advice he provided. Finally, we also agreed with Antoine (my teammate in Paris) to visit Beirut and assist me in running the first two sessions.

 

Communication 

 
After setting the start date to be on Wednesday, July 15, 2015, we sent a communication email to all Beirut employees announcing the kick-off of codingdojo and opening the door for registration. In less than one day, more than 30 participants were registered. That was a surprise!!! We weren’t expecting that many participants, especially that many of them were not developers. Since running a single session with 30 participants was impossible, we decided to divide the participants into 4 groups/sessions over the following 4 weeks.

 

First Two Sessions 

 
For the first 2 sessions, Antoine and I prepared very well by selecting and solving the problems ahead of time. Despite that, we made sure not to impose our solution, rather we tried to have an open discussion and agree with the rest on the next tests to write.

The first 30 minutes of both sessions were the hardest. Almost all participants were not familiar with the concept of writing the test before the code that is why we made sure to be the first 2 to drive the code and when someone was coding you would hear us saying “That is correct, but you forgot to write the test!” when others were coding. After that, things started to go smoother! Participants got acquainted with the concept and were aware when they missed the tests.  

For us, the first two sessions were great and much better than what we expected. We expected to have someone trying to control keyboard or dictating his solution. On the contrary, everyone respected the 5-minute limit they had. They were all sharing the solution with others. 
It was clear to us, from the feedback (listed below) we got from participants during the retrospective, that they enjoyed their time and are willing to attend future sessions:

  • Nice, good experience; you learn from others 
  • Nice technique to learn when trying to write the first test and nice way to think incrementally
  • It is hard to do it alone; it is easier with a group 
  • I am not a developer and not familiar to Java, but it was good to see the mindset of the developers
  • Nice to meet people from other teams
  • Nice practice because you are enforced to follow a convention; taught us new best practices 
  • I like the idea of refactoring; the complex code was refactored into couple lines of code that made it more readable 
  • TDD is a bit slow; the pros are the interaction between each other 
 
 

My Feedback

Running those sessions is not an easy task, it requires a lot of effort before and during each session. Before any session, the leader should prepare the right environment for the participants. During the session, the leader should always be focused on what the driver is writing and at the same time listen to suggestion proposed by others and answer the questions raised.
Despite that, it is a great experience from which you learn and gain a lot.
Later in coming sessions we might be introducing some new challenges like increasing the problem complexity or trying new programming languages. I hope that people will remain motivated and enthusiastic to attend the coming sessions!

P.S: The code we write during those sessions will be committed to this GitHub repository.