NULL Trouble In SQL Server Land

I am seeing a lot of searches for SQL + Nulls from this site so I decided to blog about it
Before I start I would like to point out that all the code will behave this way if ANSI_NULLS is set to on ,not to off

CREATE TABLE testnulls (ID INT)
INSERT INTO testnulls VALUES (1)
INSERT INTO testnulls VALUES (2)
INSERT INTO testnulls VALUES (null)

CREATE TABLE testjoin (ID INT)
INSERT INTO testjoin VALUES (1)
INSERT INTO testjoin VALUES (3)

--We get back value 1 here
SELECT * FROM testjoin WHERE ID IN(SELECT ID FROM testnulls)

--Nothing is returned
SELECT * FROM testjoin WHERE ID NOT IN(SELECT ID FROM testnulls)

--Value 3 is returned
SELECT * FROM testjoin WHERE ID NOT IN(SELECT ID FROM testnulls WHERE ID IS NOT NULL)


--value 3 is returned
SELECT * FROM testjoin j
WHERE NOT EXISTS (SELECT n.ID
FROM testnulls n
WHERE n.ID = j.ID)


--value 3 is returned
SELECT j.* FROM testjoin j
LEFT OUTER JOIN testnulls n ON n.ID = j.ID
WHERE n.ID IS NULL



--a count of 3 is returned
SELECT COUNT(*) FROM testnulls

-- a count of 2 is returned, the count ignores NULL values
SELECT COUNT(id) FROM testnulls

--By using coalesce the count is also 3
SELECT COUNT(COALESCE(id,0)) FROM testnulls

--all 3 rows are returned
SELECT * FROM testnulls

-- 1 row is returned
SELECT * FROM testnulls
WHERE ID = 1

-- only 1 row is returned the row with the NULL value is ignored
SELECT * FROM testnulls
WHERE ID <> 1


-- Now both rows that are not 1 are returned
SELECT * FROM testnulls
WHERE ID <>1
OR ID IS NULL

-- Now both rows that are not 1 are returned also
SELECT * FROM testnulls
WHERE COALESCE(ID,0) <> 1

Some more NULL Fun
You can''t compare NULL with anything
Since both values are unknown even a comparison with another NULL is unknown

DECLARE @v INT
DECLARE @v2 INT

SELECT @v =NULL,@v2 = NULL
IF @v = @v2
SELECT 'yes'
ELSE
SELECT 'No'


Be carefull with forgetting to initialize parameters while building string

DECLARE @SQL VARCHAR(500)
DECLARE @Id INT

SELECT @SQL = 'SELECT * FROM testnulls
WHERE ID =' + CONVERT(VARCHAR,@Id)

SELECT @SQL

Professional SQL Server 2005 Integration Services

Professional SQL Server 2005 Integration Services has just been published by WROX, a co-worker already ordered it and I skimmed though it during lunch and ordered my own copy. I already own SQL server 2000 DTS and was very satisfied with that book. I have included 2 links to 2 sample chapters as well as a link to Amazon, so far there are 2 reviews on Amazon and both are 5 stars. The book itself is 720 pages and below is the table of contents





Foreword.
Preface.
Acknowledegements.
Author Bios.
Introduction.

1. Services.
2. Importing and Exporting Data.
3. SSIS Fundamentals.
4. Data Flow Elements and Transforms.
5. Creating Your First Package.
6. Advanced Tasks and Transforms.
7. Scripting in DTS.
8. Accessing Heterogenous Data.
9. Reliability and Scalability.
10. Understanding the Integration Services Engine.
11. Applying the Integration Services Engine.
12. DTS 2000 Migration and Metadata Management.
13. Error and Event Handling.
14. Programming and Extending SSIS.
15. Adding a User Interface to Components.
16. External Management and WMI.
17. Using SSIS with External Applications.
18. SSIS Software Development Life Cycle.
19. Case Study: Typical Relational ETL.
Index.

Chapter 1 Welcome to SQL Server Integration Services sample chapter is available here
Chapter 5 Creating an End-to-End Package is available from Brian Knight's blog

The Amazon link is here

Format SQL Server Decimal Data

Yesterday someone posted a question in the SQL server
microsoft.public.sqlserver.programming news group about formatting decimal data. The person did not want trailing zeros or a decimal point if the amount was whole
The trick to do this is convert the decimal point to a space, rtrim it and then convert the space to a decimal. When you do that the trailing decimal point will vanish. The same applies for trailing zeros of course

output
.25000
.50000
1.00000
1.50000
2.00000
3.00000

desired output
0.25
0.5
1
1.5
2
3


--first create our test data table
CREATE TABLE #testdecimals(testdata DECIMAL(20,5))
INSERT INTO #testdecimals
SELECT 0.2500 UNION ALL
SELECT 0.5000 UNION ALL
SELECT 1.0000 UNION ALL
SELECT 1.5000 UNION ALL
SELECT 2.0000 UNION ALL
SELECT 3.0000

-- regular select
SELECT testdata
FROM #testdecimals

--formatted select
SELECT REPLACE(RTRIM(REPLACE(REPLACE(RTRIM(REPLACE(CONVERT(varchar,testdata),'0',' ')),' ','0'),'.',' ')),' ','.')
FROM #testdecimals

Use XP_CMDSHELL To Get All File Names With Size In A Directory

To get all the files with their filesize in a directory use xp_cmdshell with a temporary table
After the temporary table is populated you will be able to sort the result set any way you like


CREATE TABLE #tempList (Files VARCHAR(500))

INSERT INTO #tempList
EXEC MASTER..XP_CMDSHELL 'dir c:\ '


--delete all directories
DELETE #tempList WHERE Files LIKE '%<dir>%'

--delete all informational messages
DELETE #tempList WHERE Files LIKE ' %'

--delete the null values
DELETE #tempList WHERE Files IS NULL

--get rid of dateinfo
UPDATE #tempList SET files =RIGHT(files,(LEN(files)-20))

--get rid of leading spaces
UPDATE #tempList SET files =LTRIM(files)

--split data into size and filename
SELECT LEFT(files,PATINDEX('% %',files)) AS Size,
RIGHT(files,LEN(files) -PATINDEX('% %',files)) AS FileName
FROM #tempList

Login failed for user 'sa'. Reason: Not associated with a trusted SQL Server connection. SQL 2000

I am seeing a ton of searches on this blog or reaching this blog from MSN/Google with this search “Login failed for user 'sa'. Reason: Not associated with a trusted SQL Server connection.” So this is what causes the problem: your SQL Server has been setup with windows authentication only. In order to make it mixed mode authentication right click on the server name in enterprise manager select properties and click on the security tab. Select SQL server and Windows (see picture…) and that will fix it
I will monitor the searches and write more stuff about things that people are searching for.
For the SQL server 2005 version go here

Kill All Connections To Your SQL Server Database

Sometimes you need to restore a database and put it in single user mode but you can't because people are still using it. I will show you 2 ways to accomplish that
The first way is to loop through the sysprocesses table and kill all the processes one by one. The second way is to do an alter statement on the DB


--loop through the sysprocesses
DECLARE @sysDbName SYSNAME
SELECT @sysDbName = 'northwind'

SELECT IDENTITY(int, 1,1)AS ID,spid
INTO #LoopProcess
FROM master..sysprocesses
WHERE dbid = DB_ID(@sysDbName)


DECLARE @SPID SMALLINT
DECLARE @SQL VARCHAR(255)
DECLARE @MaxID INT, @LoopID INT

SELECT @LoopID =1,@MaxID = MAX(ID) FROM #LoopProcess

WHILE @LoopID <= @MaxID
BEGIN
SELECT @SPID = spid FROM #LoopProcess WHERE ID = @LoopID
SELECT @SQL = 'KILL ' + CONVERT(VARCHAR, @SPID)

EXEC( @SQL )

SELECT @LoopID = @LoopID +1
END

DROP TABLE #LoopProcess


--alter the DB by making it single user (all transaction will be rolled back)
ALTER DATABASE northwind SET SINGLE_USER WITH ROLLBACK IMMEDIATE

--do your restore here


-- Make the DB multi user again
ALTER DATABASE northwind SET MULTI_USER

Improve Your SQL Server Skills By Helping Others

Participating in forums and newsgroups is a great way to improve your SQL Server skills. You will see other people's problems and if you are just starting out then just relax and read the responses. The responses are most likely from a SQL Server MVP and these are top-quality. You will see many familiar authors in the MS public newsgroups such as Kalen Delaney, Itzik Ben-Gan, Adam Machanic, Louis Davidson and more. I have encountered SQL Server problems and remembered that I read about this on a SQL Server newsgroup and that a person posted a solution, from then on it's quick Google search and there you have it.

MS Public SQL Server Newsgroups
microsoft.public.sqlserver.newusers
microsoft.public.sqlserver.programming
microsoft.public.sqlserver.server

And a full list here

BTW these newsgroups can also be accessed by a newsreader as NTTP

Besides the newsgroups I also participate in the SQL Server Tek-Tip Forums for programming and administration What's great about Tek-Tips is the fact that people give stars to people who post a useful solution. So this will give you more incentives to post a good solution. You can also write searchable FAQ's

Tek-Tip SQL Server Forums
programming
administration

Like I said before you don't have to post solution but do at least spend some of your time reading the solutions to people's problems because sooner or later you might encounter them yourself.

sp_MSforeachtable

Find out fragmentation levels for all tables in your database by running the undocumented stored procedure sp_MSforeachtable

-- Create temp table
CREATE TABLE #fraglist (
ObjectName CHAR (255),
ObjectId INT,
IndexName CHAR (255),
IndexId INT,
Lvl INT,
CountPages INT,
CountRows INT,
MinRecSize INT,
MaxRecSize INT,
AvgRecSize INT,
ForRecCount INT,
Extents INT,
ExtentSwitches INT,
AvgFreeBytes INT,
AvgPageDensity INT,
ScanDensity DECIMAL,
BestCount INT,
ActualCount INT,
LogicalFrag DECIMAL,
ExtentFrag DECIMAL)

--Insert fragmentation level information
sp_MSforeachtable 'insert into #fraglist exec(''DBCC showcontig (''''?'''') with tableresults '') '

--Get all the info back
SELECT * FROM #fraglist

After you have run this you can determine which tables should be defragmented by looking at the AvgFreeBytes, ScanDensity, LogicalFrag and ExtenFrag columns

Find All Tables Without Triggers In SQL Server

Sometimes you want to change tables by adding columns, dropping or changing column types
You want to make sure that there aren't any tables that have triggers on them so that stuff doesn't start to break after you make these changes
I present two ways to accomplish that, the first way is by joining the INFORMATION_SCHEMA.TABLES view with the sysobjects system table.
The second way is to do a self join on the sysobjects system table. Both of these queries will return the same result


SELECT t.TABLE_NAME FROM INFORMATION_SCHEMA.TABLES t
LEFT JOIN(SELECT OBJECT_NAME(o.parent_obj) AS TableName
FROM sysobjects o
WHERE OBJECTPROPERTY(o.[id], 'IsTrigger') = 1
) tr ON t.TABLE_NAME= tr.TableName
WHERE tr.TableName IS NULL
AND TABLE_TYPE ='BASE TABLE'
ORDER BY t.TABLE_NAME

SELECT s1.name FROM sysobjects s1 LEFT JOIN sysobjects s2 ON
s1.id =s2.parent_obj
AND s2.xtype = 'TR'
WHERE s2.name IS NULL
AND s1.xtype = 'U'
ORDER BY s1.name

ISNUMERIC SQL Server Trouble

If you use the ISNUMERIC function to determine if a value is numeric you might be in for a surprise
Run the 3 lines of code below and you will see what I mean

DECLARE @S VARCHAR(50)
SET @S = CHAR(9) --- @S NOW CONTAINS THE TAB CHARACTER
SELECT ISNUMERIC(@S), ISNUMERIC(CHAR(9)),ISNUMERIC('1D2'),ISNUMERIC('D')

As you can see TAB is returned as numeric as well the value 1D2
A better way to test for this would be with LIKE and %[a-z]%
If you run the example below you will see that the select statement with the ISNUMERIC function or LIKE returns one row more than the statement with LIKE and ISNUMERIC combined

CREATE TABLE #foo (Value VARCHAR(20))
INSERT INTO #foo
SELECT '1' UNION ALL
SELECT '3' UNION ALL
SELECT 'B' UNION ALL
SELECT '2' UNION ALL
SELECT '33.331' UNION ALL
SELECT 'adad1' UNION ALL
SELECT '1d2' UNION ALL
SELECT '^' UNION ALL
SELECT '17777.999'

--returns ^
SELECT * FROM #foo
WHERE Value NOT LIKE '%[a-z]%'

--returns 1d2
SELECT * FROM #foo
WHERE ISNUMERIC(Value) = 1

--returns correct result
SELECT * FROM #foo
WHERE Value NOT LIKE '%[a-z]%'
AND ISNUMERIC(Value) = 1

Compare Tables With Tablediff

In SQL Server 2005 there is a new utility to compare the data in two tables these can be in different tables and different servers.
The utility is Tablediff

The location is Program Files\Microsoft SQL Server\90\COM\tablediff.exe

From BOL:

The tablediff utility is used to compare the data in two tables for non-convergence, and is particularly useful for troubleshooting non-convergence in a replication topology. This utility can be used from the command prompt or in a batch file to perform the following tasks:

A row by row comparison between a source table in an instance of Microsoft SQL Server acting as a replication Publisher and the destination table at one or more instances of SQL Server acting as replication Subscribers.

Perform a fast comparison by only comparing row counts and schema.

Perform column-level comparisons.

Generate a Transact-SQL script to fix discrepancies at the destination server to bring the source and destination tables into convergence.

Log results to an output file or into a table in the destination database.

Look it up in Books On Line

Troubleshooting Performance Problems in SQL Server 2005

It is not uncommon to experience the occasional slow down of a SQL Server database. A poorly designed database or a system that is improperly configured for the workload are but several of many possible causes of this type of performance problem. Administrators need to proactively prevent or minimize problems and, when they occur, diagnose the cause and take corrective actions to fix the problem. This article provides step-by-step guidelines for diagnosing and troubleshooting common performance problems by using publicly available tools such as SQL Server Profiler, System Monitor, and the new Dynamic Management Views in SQL Server 2005
There is also a word document versionf for download
If you are a SQL Server 2005 developer/administrator then this is a must read

You will be introduced to new views such as sys.dm_exec_sql_text, sys.dm_exec_cached_plan, Sys.dm_db_index_operational_stats and sys.dm_db_index_usage_stats

Below is a short list of what is covered in the article

Resource Bottlenecks
Tools for resolving resource bottlenecks

CPU Bottlenecks
Excessive compilation and recompilation
Inefficient query plan
Intra-query parallelism
Poor cursor usage


Memory Bottlenecks
External physical memory pressure
External virtual memory pressure
Internal physical memory pressure
Caches and memory pressure
Ring buffers
Internal virtual memory pressure



Tempdb
Monitoring tempdb space
Troubleshooting space issues
User objects
Version store
Internal Objects
Excessive DDL and allocation operations
Resolution

Slow-Running Queries
Blocking
Identifying long blocks
Blocking per object with sys.dm_db_index_operational_stats
Overall performance effect of blocking using SQL waits
Monitoring index usage


Appendix A: DBCC MEMORYSTATUS Description
Appendix B: Blocking Scripts


Read the whole article here (Troubleshooting Performance Problems in SQL Server 2005)

Contact Me

If you want to contact me and let me know that you like this blog or that you hate it or if you have any kind of suggestion then run the code below in Query Analyzer/Managment Studio to get my email (to prevent spam)

SELECT CONVERT(VARCHAR(31),0x73716C736572766572636F646540676D61696C2E636F6D)

Top 5 SQL Server Posts for December 2005

Below are the top 5 posts according to Google Analytics for the month of December

1) Feature Pack for Microsoft SQL Server 2005
2) Fun With SQL Server Update Triggers
3) SQL Server 2005 Free E-Learning
4) Find all Primary and Foreign Keys In A Database
5) Pad Numbers By Using CASE, LEFT And CONVERT

SQL Server Site Geo Map For December

These are the countries where the bulk of the visitors for this blog are coming from. As you can see most of the visitors are from the US and Europe, Asia is in third place. When I look at the maps day by day I also see some visitors from Africa and the Middle East but you need more than one visitor per city in order for it to show up in the monthly map. I only started Google Analytics at the end of November To see the November geo map click here

Top SQL Server Google Searches For December

These are the top SQL Searches on this site for the month of December. I have left out searches that have nothing to do with SQL Server or programming. Here are the results...
CONCATENATION
dts append
general network error
"Remote Server" "Linked Server" MSSQL
TOP command in SQL2000
append
OLE DB provider 'SQLOLEDB' reported an error. The provider ran out of memory.
delete trigger

Happy New Year

Happy new year to all of you