How to disable tests in tSQLt

The tSQLt unit test framework that we use to do our DB tests does not allow for disabled tests, this is because each test is just a stored proc. Right now the only way is to either rename the proc so it doesn’t start with test or to remove the test

I was thinking about this in the shower this morning….here is what I came up with

Here is a quick way how to prevent a test from running when the tSQLt tests run.
Put the code below into each unit test that you want disabled, you should put this code right after the CREATE PROCEDURE  ProcName AS part

This code will pick the correct name and schema, so you can use the same exact code in each test

DECLARE @IsTestDisabled bit = 1 -- set this to 1 if you don't want the test to run

IF @IsTestDisabled =1
BEGIN
 DECLARE @SchemaName varchar(1000)
 SELECT @SchemaName = QUOTENAME(SCHEMA_NAME(schema_id)) +'.'
 FROM sys.procedures WHERE object_id = @@procid

 PRINT 'Disabled Test: ' + @SchemaName +  QUOTENAME(OBJECT_NAME(@@procid))
 RETURN
END

So if we grab the example test from the tSQLt website

We would change it from this

CREATE PROCEDURE testFinancialApp.[test that ConvertCurrency converts using given conversion rate]
AS


BEGIN
    DECLARE @actual MONEY;
    DECLARE @rate DECIMAL(10,4); SET @rate = 1.2;
    DECLARE @amount MONEY; SET @amount = 2.00;

    SELECT @actual = FinancialApp.ConvertCurrency(@rate, @amount);

    DECLARE @expected MONEY; SET @expected = 2.4;   --(rate * amount)
    EXEC tSQLt.AssertEquals @expected, @actual;

END;
GO

To  this

CREATE PROCEDURE testFinancialApp.[test that ConvertCurrency converts using given conversion rate]
AS

DECLARE @IsTestDisabled bit = 1 -- set this to 1 if you don't want the test to run

IF @IsTestDisabled =1
BEGIN
 DECLARE @SchemaName varchar(1000)
 SELECT @SchemaName = QUOTENAME(SCHEMA_NAME(schema_id)) +'.'
 FROM sys.procedures WHERE object_id = @@procid

 PRINT 'Disabled Test: ' + @SchemaName +  QUOTENAME(OBJECT_NAME(@@procid))
 RETURN
END


BEGIN
    DECLARE @actual MONEY;
    DECLARE @rate DECIMAL(10,4); SET @rate = 1.2;
    DECLARE @amount MONEY; SET @amount = 2.00;

    SELECT @actual = FinancialApp.ConvertCurrency(@rate, @amount);

    DECLARE @expected MONEY; SET @expected = 2.4;   --(rate * amount)
    EXEC tSQLt.AssertEquals @expected, @actual;

END;
GO

Now you will get the following in your tSQLt test run output

Disabled Test: [testFinancialApp].[test that ConvertCurrency converts using given conversion rate]

This will be right above this text

+----------------------+
|Test Execution Summary|
+----------------------+

I want this printed so we know we have disabled tests when we get a email whenever Jenkins kicks off these tests…..


To enable the test again, all you have to do is change the value from 1 to 0 here. So instead of

DECLARE @IsTestDisabled bit = 1

You would make it

DECLARE @IsTestDisabled bit = 0

To see all my tSQLt posts, visit this link: http://sqlservercode.blogspot.com/search/label/tSQLt

Some reasons your tSQLt test is not executing

This post is part of a Unit Testing series I created, you can find links to all the posts here: Unit testing in SQL Server, a list of all the posts
Today we are going to look at why a test might not be running. Before starting, if you don't have tSQLt setup, follow the instruction here: Unit testing in SQL Server, installing tSQLt and doing a quick test
 First, if you didn't create a test database yet, let's create one and name it UnittestExamples, we will also create a simple table in that database

CREATE DATABASE UnittestExamples
GO

USE UnittestExamples
GO

CREATE TABLE SomeTable (ID int)

Now connect to your tSQLt database
USE tSQLt
GO

Let's create two test classes

EXEC tSQLt.NewTestClass 'MyNewClass'
EXEC tSQLt.NewTestClass 'MyNewestClass'

To see all the test classes in your tSQLt database, you can use the following query

SELECT Name, SchemaId
FROM tSQLt.TestClasses;

Name       SchemaId
MyNewClass 6
MyNewestClass 7

Now we will create a bunch of procs, these procs will test for the existance of a table in the UnittestExamples database

CREATE PROCEDURE MyNewClass.[ObjectExist]
AS
EXEC tSQLt.AssertObjectExists 'UnittestExamples.dbo.SomeTable'
GO

CREATE PROCEDURE MyNewestClass.[ObjectExist]
AS
EXEC tSQLt.AssertObjectExists 'UnittestExamples.dbo.SomeTable'
GO



CREATE PROCEDURE MyNewClass.[test ObjectExist]
AS
EXEC tSQLt.AssertObjectExists 'UnittestExamples.dbo.SomeTable'
GO

CREATE PROCEDURE MyNewestClass.[test ObjectExist]
AS
EXEC tSQLt.AssertObjectExists 'UnittestExamples.dbo.SomeTable'
GO


CREATE PROCEDURE [test ObjectExist]
AS
EXEC tSQLt.AssertObjectExists 'UnittestExamples.dbo.SomeTable'
GO

Now it is time to run our tests,

EXEC tSQLt.tSQLt.Run 'MyNewestClass'

+----------------------+
|Test Execution Summary|
+----------------------+

|No|Test Case Name                    |Result |
+--+----------------------------------+-------+
|1 |[MyNewestClass].[test ObjectExist]|Success|
-----------------------------------------------------------------------------
Test Case Summary: 1 test case(s) executed, 1 succeeded, 0 failed, 0 errored.
-----------------------------------------------------------------------------

EXEC tSQLt.tSQLt.Run 'MyNewClass'

+----------------------+
|Test Execution Summary|
+----------------------+

|No|Test Case Name                 |Result |
+--+-------------------------------+-------+
|1 |[MyNewClass].[test ObjectExist]|Success|
-----------------------------------------------------------------------------
Test Case Summary: 1 test case(s) executed, 1 succeeded, 0 failed, 0 errored.
-----------------------------------------------------------------------------

Now, let;s call all the classes in one shot, will that make a difference?

EXEC tSQLt.tSQLt.RunAll

+----------------------+
|Test Execution Summary|
+----------------------+

|No|Test Case Name                 |Result |
+--+-------------------------------+-------+
|1  |[MyNewClass].[test ObjectExist]    |Success|
|2  |[MyNewestClass].[test ObjectExist] |Success|
-----------------------------------------------------------------------------
Test Case Summary: 2 test case(s) executed, 2 succeeded, 0 failed, 0 errored.
-----------------------------------------------------------------------------

Nope, only two out of five tests executed.
So what happened, why are only two of the five tests running? The reason is that each test has to be named so that the first part of the name is test. We created the following four test stored procedures

CREATE PROCEDURE MyNewClass.[ObjectExist]
CREATE PROCEDURE MyNewestClass.[ObjectExist]

CREATE PROCEDURE MyNewClass.[test ObjectExist]
CREATE PROCEDURE MyNewestClass.[test ObjectExist]

As you can see the first two are not named so that the name starts with test. Renaming those so that they start with test will fix that. I myself have made this mistake many times in the beginning, hopefully this will save someone some time in the future

What about the 5th proc why didn't that run?

CREATE PROCEDURE [test ObjectExist]

As you can see that proc is named correctly, however it is not in the two test classes that we created. It is created in the default class which most likely will be dbo, You can actually execute this by calling the dbo class

EXEC tSQLt.tSQLt.Run 'dbo'

+----------------------+
|Test Execution Summary|
+----------------------+

|No|Test Case Name          |Result |
+--+------------------------+-------+
|1 |[dbo].[test ObjectExist]|Success|
-----------------------------------------------------------------------------
Test Case Summary: 1 test case(s) executed, 1 succeeded, 0 failed, 0 errored.
-----------------------------------------------------------------------------


So there you have it, the procs have to be named so that they start with test and the procs have to be created in a test class that the following query returns

SELECT Name, SchemaId
FROM tSQLt.TestClasses;

Time to clean up the mess...... Here is how you can quickly clean up, this will drop the test class as well as all the objects in that class

EXEC tSQLt.DropClass 'MyNewClass'
EXEC tSQLt.DropClass 'MyNewestClass'

This post is part of a Unit Testing series I created, you can find links to all the posts here: Unit testing in SQL Server, a list of all the posts

Unit testing in SQL Server, a list of all the posts

This is a list of all the unit test blog post I created about tSQLt and unit testing.

Unit testing in SQl Server... why?
This post explains why you should be unit testing

Unit testing in SQL Server..what are unit tests
This post explains what unit tests are

Unit testing in SQL Server, taking a first look a tSQLt
This post takes a first look at tSQLt, tSQLt is a unit test framework completely written in T-SQL

Unit testing in SQL Server, installing tSQLt and doing a quick test
This post helps you install tSQLt and run a few test to get a quick demo of how tSQLt works

Some reasons your tSQLt test is not executing
This post will explain why some unit tests might not be running

Unit testing in SQL Server, installing tSQLt and doing a quick test

In today's post we will install tSQLt and run a simple test. To start download tSQLt, unzip the zipfile into its own folder. Here is what is inside that zip file, we are looking at executing the tSQLt.class.sql file later on

Open up SSMS, connect to the database server where you want to create the tSQLt database. We need to enable the CLR on the server where you are running your tests because tSQLt is using some CLR functionality.

Execute the following command to enable the CLR on your box

EXEC sp_configure 'clr enabled', 1;
RECONFIGURE;

Now we need to create the database and make it trustworthy, execute the following

CREATE DATABASE tSQLt
GO

USE tSQLt
GO

ALTER DATABASE tSQLt SET TRUSTWORTHY ON;
GO

Go to your unzipped folder, open up the tSQLt.class.sql file, make sure that you are connected to you tSQLt database, execute the code

You should see something like this in the output once all the code has run

Installed at 2015-10-05 17:58:53.360

(1 row(s) affected)

+-----------------------------------------+
|                                         |
| Thank you for using tSQLt.              |
|                                         |
| tSQLt Version: 1.0.5686.18945           |
|                                         |
+-----------------------------------------+

After the code has completed, run the following

EXEC tSQLt.Fail 'congratulation, we are ready to go to the next step'

You should see the following output

Msg 50000, Level 16, State 10, Procedure Fail, Line 51
tSQLt.Failure


Let's create a sample database so that we can do some testing

Execute the following code

CREATE DATABASE UnittestExamples
GO

USE UnittestExamples
GO

CREATE TABLE dbo.test1(id int)
CREATE TABLE dbo.test2(id int)

INSERT dbo.test1  VALUES(1)
INSERT dbo.test2  VALUES(2)

INSERT dbo.test1  VALUES(3)
INSERT dbo.test2  VALUES(3)

INSERT dbo.test1  VALUES(4)
INSERT dbo.test2  VALUES(5)
GO

USE tSQLt
GO

Now let's run a couple of tests

tSQLt ships with an assertion to test if 2 tables are the same, the assertion is named AssertEqualsTable

So let's see if these two tables are the same, execute the following

EXEC tSQLt.AssertEqualsTable 'UnittestExamples.dbo.test1', 'UnittestExamples.dbo.test2'

Here is the output
Msg 50000, Level 16, State 10, Procedure Fail, Line 73
tSQLt.Failure

As you just saw, will get an error, now what will happen if you compare the table against itself, run this to find out.

EXEC tSQLt.AssertEqualsTable 'UnittestExamples.dbo.test1', 'UnittestExamples.dbo.test1'

There was no error now, this is expected

What happens if we test a table against a non existing table?

EXEC tSQLt.AssertEqualsTable 'UnittestExamples.dbo.test1', 'UnittestExamples.dbo.test112'

Msg 50000, Level 16, State 10, Procedure Fail, Line 51
tSQLt.Failure
Msg 208, Level 16, State 1, Line 2
Invalid object name 'UnittestExamples.dbo.test112'.

I want to point something important out. Comparing two tables is usually doing for small lookup tables, you create a table inside your unit test with values that you expect to match the table in the database, if these tables don't match, then you have a problem. You wouldn't want to compare two million row tables in a test.


Let's do another assertion, let's see if an table exists.

EXEC tSQLt.AssertObjectExists 'UnittestExamples.dbo.SomeTable'

Msg 50000, Level 16, State 10, Procedure Fail, Line 51
tSQLt.Failure

That fails as expected, now let's try it with a table that exists

EXEC tSQLt.AssertObjectExists 'UnittestExamples.dbo.test1'

That ran fine as expected.

Let's clean up by dropping the two tables we created earlier.

DROP TABLE dbo.test1 , dbo.test12

Creating a test class
What you want to do with tSQLt is group all your tests together, for example if you are testing a stored procedure, you can execute just those set of tests. When creating a test class in tSQLt, a new schema will be created, you can easily verify this. Execute the following

USE tSQLt
GO

SELECT * FROM sys.schemas
WHERE schema_id BETWEEN 5 AND 1000

name schema_id principal_id
tSQLt         5 1

As you can see, the only schema the query returns is the tSQLt schema. Now let's create a new test class, you do this by executing the NewTestClass  stored procedure, if we wanted to create a test class named WebFlags, we would do it like this

tSQLt.NewTestClass 'WebFlags'


Running the following code again, you will now see an additional schema in the output

USE tSQLt
GO

SELECT * FROM sys.schemas
WHERE schema_id BETWEEN 5 AND 1000

name schema_id principal_id
tSQLt         5 1
WebFlags 6 1  


Let's create a couple of test stored procedures with the examples we used before

First we need to recreate the test1 table again

USE UnittestExamples
GO

CREATE TABLE dbo.test1(id int)


INSERT dbo.test1  VALUES(1)
INSERT dbo.test1  VALUES(3)
INSERT dbo.test1  VALUES(4)
GO

Now we will create 3 stored procedures. It is very important that you name each stored procedure beginning with 'test ', if you don't do this then tSQLt will skip the proc. Here is the code to create our 3 stored procedures

USE tSQLt
GO

CREATE PROCEDURE WebFlags.[test test1 table has all correct values]
AS

CREATE TABLE dbo.test2(id int)

INSERT dbo.test2  VALUES(2)
INSERT dbo.test2  VALUES(3)
INSERT dbo.test2  VALUES(5)

EXEC tSQLt.AssertEqualsTable 'UnittestExamples.dbo.test1', 'tSQLt.dbo.test2'

DROP TABLE dbo.test2
GO



CREATE PROCEDURE WebFlags.[test SomeTable table exists]
AS
EXEC tSQLt.AssertObjectExists 'UnittestExamples.dbo.SomeTable'
GO

CREATE PROCEDURE WebFlags.[test test1 table exists]
AS
EXEC tSQLt.AssertObjectExists 'UnittestExamples.dbo.test1'
GO

To execute these procs, all we need to do is run the following

tSQLt.tSQLt.Run 'WebFlags'

Here is the output

[WebFlags].[test SomeTable table exists] failed: (Failure) 'UnittestExamples.dbo.SomeTable' does not exist
[WebFlags].[test test1 table has all correct values] failed: (Failure) Unexpected/missing resultset rows!
|_m_|id|
+---+--+
|<  |1 |
|>  |4 |
|=  |3 |
|>  |5 |
|>  |2 |

+----------------------+
|Test Execution Summary|
+----------------------+

|No|Test Case Name                                      |Dur(ms)|Result |
+--+----------------------------------------------------+-------+-------+
|1 |[WebFlags].[test test1 table exists]                |      3|Success|
|2 |[WebFlags].[test SomeTable table exists]            |      6|Failure|
|3 |[WebFlags].[test test1 table has all correct values]|     33|Failure|
-----------------------------------------------------------------------------
Msg 50000, Level 16, State 10, Line 1
Test Case Summary: 3 test case(s) executed, 1 succeeded, 2 failed, 0 errored.
-----------------------------------------------------------------------------



As you can see, on top you will get the name of the tests that failed, also included is the error message

[WebFlags].[test SomeTable table exists] failed: (Failure) 'UnittestExamples.dbo.SomeTable' does not exist
[WebFlags].[test test1 table has all correct values] failed: (Failure) Unexpected/missing resultset rows!
|_m_|id|
+---+--+
|<  |1 |
|>  |4 |
|=  |3 |
|>  |5 |
|>  |2 |

Below that you will get a list of each that that ran with an indication if the test was successful or not

|No|Test Case Name                                      |Dur(ms)|Result |
+--+----------------------------------------------------+-------+-------+
|1 |[WebFlags].[test test1 table exists]                |      3|Success|
|2 |[WebFlags].[test SomeTable table exists]            |      6|Failure|
|3 |[WebFlags].[test test1 table has all correct values]|     33|Failure|

Finally you get a summary

Test Case Summary: 3 test case(s) executed, 1 succeeded, 2 failed, 0 errored.


You can also run all these tests by running the following

tSQLt.tSQLt.RunAll

Right now since you only have 1 test class, the result is the same if you run all tests or the tests for the WebFlags class

That's all for today, in the next post we will take a look at tSQLt available assertions and how they are used

The list of all the unit test posts can be found here: Unit testing in SQL Server, a list of all the posts

Unit testing in SQL Server, taking a first look a tSQLt

This is the third post in my Unit Testing series, today we are looking at what tSQLt, in the next post we will take a look at how to install tSQLt

What is tSQLt?  Here is how it is described on the tSQLt website


tSQLt is a database unit testing framework for Microsoft SQL Server. tSQLt is compatible with SQL Server 2005 (service pack 2 required) and above on all editions.

Main features

tSQLt allows you to implement unit tests in T-SQL. This is important as you do not have to switch between various tools to create your code and your unit tests. tSQLt also provides the following features to make it easier to create and manage unit tests:

• Tests are automatically run within transactions – this keeps tests independent and reduces any cleanup work you need
• Tests can be grouped together within a schema – allowing you to organize your tests and use common setup methods
• Output can be generated in plain text or XML – making it easier to integrate with a continuous integration tool
• Provides the ability to fake tables and views, and to create stored procedure spies – allowing you to isolate the code which you are testing

So basically after you install tSQLt on your SQL Server instance, it is just a database, you now have a unit test framework that you can use. They way you use tSQLt is by creating stored procedures that will do the unit testing for you. It all runs within SQL Server.

You can run 1 test, all tests in a test class or all tests for all test classes by calling a single stored procedure.  Here are the Assertions and Expectations that ship with tSQLt

Assertions

AssertEmptyTable

AssertEquals

AssertEqualsString

AssertEqualsTable

AssertLike

AssertNotEquals

AssertObjectDoesNotExist

AssertObjectExists

AssertResultSetsHaveSameMetaData

Fail

Expectations

ExpectException

ExpectNoException

In addition to that you can of course use IF EXISTS..... and then a Fail Assertions to test anything that is not testable with the built in Assertions 

In addition to be able to run this all from a query window in SSMS, if you have Red Gates SQL Test then you can use that to run the tests, you will get a graphical indicator at that point. If the test passes it will be green, if it fails it will be red.

In the next post, we will take a look at how to install tSQLt on your database instance and write our  first test.

Unit testing in SQL Server..what are unit tests

This is a continuation from yesterday's post Unit testing in SQl Server... why? Today we will look at what unit tests actually are.

Test on an individual unit of code.
A unit test should ideally only test one discrete unit of code, there should not be a test for example how this unit of code reacts with other units of code. Tests should not be influenced by other units.

Repeatable
Executing a unit test should give you the same result every single time. Since a unit test is repeatable it is also easy to be automated.

Test one thing only
A test should only be testing one thing, it should test one question only. For example if I put money in the account did the balance increase?

Test the requirement for the unit.
If the requirement is that zip codes are 5 digits for example then you should have tests that verify that 5 is correct and for example 4 and 6 will fail the test.



When do I write unit tests?

Before development?
This enables you to focus on requirements, it is part of Agile, it also allows you to develop code structures first

During development?
If requirements change you need to create new unit tests or modify existing unit tests

After development?
You will need to add tests that validate existing code


The answer is of course during all 3 phases!
You will be writing unit tests when you change something in an existing application, start a new application or if you are in the middle of development for a soon to be launched application. The only difference is that with a new application you will write the test before the code and with an existing application that you will write the unit test after the code already exists.




Why are you not unit testing yet?
I asked this question and here are some of the answers I got...I hope the last one is a joke  :-)


Don’t know where to start
No time
No benefit
Our customer do our unit testing for us

Hopefully once you are done reading all these posts, these answers should not prevent you from starting to do unit testing

Unit testing in SQl Server... why?

I have been using tSQLt to do unit testing within SQL Server for the last 2 years. I already did a presentation twice at work and decided to document some of this so that it might help other people looking at using tSQLt to do unit testing with SQL Server.

This will be a multi part series, some of the stuff I will cover:

What is unit testing and why you should be unit testing
What tSQLt is and how to get started
Sample of all the different functionality that tSQLt provides
How to integrate tSQLt within your builds


Before starting, let me first tell you what happens when something is checked in into out source code repository.
1) The applications are build
2) Integration tests are run
3) Unit tests are run
4 If there were any errors an email is sent to the developers with the output of the failed tests or compilation errors

If there are errors, tSQLt will give you the name and the error for each test as well have a summary which looks something like the following

---------------------------------------------------------------------------------
Test Case Summary: 128 test case(s) executed, 127 succeeded, 1 failed, 0 errored.
---------------------------------------------------------------------------------


Before explaining what unit tests are, let take a look at what the cost is of fixing a bug

Take a look at this image below from a presentation by Stefan Priebsch, it shows you what the cost is of fixing a bug within the software lifecycle

As you can see, the relative cost fixing a bug is 150 times compared to if it was fixed during the requirements stage.

When doing unit testing, you hopefully will catch the errors before it gets to the point where the cost to fix the bug is 20x or higher.


In the next post I will explain what unit tests are and what they will provide to you and your team